Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
J9, OMR and beyond
Serviceable Runtimes
• Bjørn Vårdal
• IBM Runtimes Developer
• J9 JVM
• @bvaardal
• MethodHandles
• Lambdas
• VarHandles
• Project Valhalla
Bjø...
Bjørn Vårdal IBM Runtime Technologies 3
Most die young_______runtimes
unless you service them.
The information contained in this presentation is provided for informational purposes only.
Whilst efforts were made to ve...
• Advanced core dump analysis for J9
• Used by the J9 service team
• Used by J9 developers
• Front-end: DDR Interactive
Bj...
$ java -cp j9ddr.jar com.ibm.j9ddr.tools.ddrinteractive.DDRInteractive core.12345
Bjørn Vårdal IBM Runtime Technologies 6
...
$ java -cp j9ddr.jar com.ibm.j9ddr.tools.ddrinteractive.DDRInteractive core.12345
Bjørn Vårdal IBM Runtime Technologies 7
...
$ java -cp j9ddr.jar com.ibm.j9ddr.tools.ddrinteractive.DDRInteractive core.12345
*0 : PID: 2824; !j9javavm 0x7f27480eef0
...
$ java -cp j9ddr.jar com.ibm.j9ddr.tools.ddrinteractive.DDRInteractive core.12345
*0 : PID: 2824; !j9javavm 0x7f27480eef0
...
$ java -cp j9ddr.jar com.ibm.j9ddr.tools.ddrinteractive.DDRInteractive core.12345
*0 : PID: 2824; !j9javavm 0x7f27480eef0
...
$ java -cp j9ddr.jar com.ibm.j9ddr.tools.ddrinteractive.DDRInteractive core.12345
*0 : PID: 2824; !j9javavm 0x7f27480eef0
...
$ java -cp j9ddr.jar com.ibm.j9ddr.tools.ddrinteractive.DDRInteractive core.20170315.010552.2824.01.dmp
*0 : PID: 2824; !j...
$ java -cp j9ddr.jar com.ibm.j9ddr.tools.ddrinteractive.DDRInteractive core.20170315.010552.2824.01.dmp
*0 : PID: 2824; !j...
$ java -cp j9ddr.jar com.ibm.j9ddr.tools.ddrinteractive.DDRInteractive core.20170315.010552.2824.01.dmp
*0 : PID: 2824; !j...
$ java -cp j9ddr.jar com.ibm.j9ddr.tools.ddrinteractive.DDRInteractive core.20170315.010552.2824.01.dmp
*0 : PID: 2824; !j...
> !stack 0x0228d70
<1a2970> Generic special frame
<1a2970> !j9method 0x01B9A718 X.print(Ljava/lang/String;)V
<1a2970> !j9m...
> !stack 0x0228d70
<1a2970> Generic special frame
<1a2970> !j9method 0x01B9A718 X.print(Ljava/lang/String;)V
<1a2970> !j9m...
> !stackslots 0x0228d70
<1a2970> *** BEGIN STACK WALK, flags = 0401 walkThread = 0x01A2970 ***
<1a2970> ITERATE_O_SLOTS
<1...
> !stackslots 0x0228d70
<1a2970> *** BEGIN STACK WALK, flags = 0401 walkThread = 0x01A2970 ***
<1a2970> ITERATE_O_SLOTS
<1...
> !stackslots 0x0228d70
<1a2970> *** BEGIN STACK WALK, flags = 0401 walkThread = 0x01A2970 ***
<1a2970> ITERATE_O_SLOTS
<1...
> !stackslots 0x0228d70
<1a2970> *** BEGIN STACK WALK, flags = 0401 walkThread = 0x01A2970 ***
<1a2970> ITERATE_O_SLOTS
<1...
> !j9object 0x0FFEF46D0
J9VMJavaLangString at 0x00000000FFEF46D0 {
struct J9Class* clazz = !j9class 0x1A5B500 // java/lang...
> !j9object 0x0FFEF46D0
J9VMJavaLangString at 0x00000000FFEF46D0 {
struct J9Class* clazz = !j9class 0x1A5B500 // java/lang...
GDB
• Requires intimate knowledge
about the JVM internal structures
• Requires debug symbols
• Source must match
• Platfor...
> !stackslots 0x0228d70
<1a2970> *** BEGIN STACK WALK, flags = 0401 walkThread = 0x01A2970 ***
<1a2970> ITERATE_O_SLOTS
<1...
Bjørn Vårdal IBM Runtime Technologies 26
DDR Interactive
”The tool that I use the most and that saves me the most time.”
J...
• We are open-sourcing the J9 JVM
• DDR coming along for the ride 
• http://openj9.org
Bjørn Vårdal IBM Runtime Technolog...
Reliable Shared Components for High Performance Language Runtimes
http://www.eclipse.org/omr
https://github.com/eclipse/om...
Bjørn Vårdal IBM Runtime Technologies 29
Agenda
• DDR in J9
• Capabilities
• How DDR is implemented
• OMR ddrgen
• Generat...
• Print structure at address
!j9object 0x12345678
!j9method 0xABCDEF00
• Other printing commands
!stackslots 0xABCDEF0
• U...
• Print structure at address
!j9object 0x12345678
!j9method 0xABCDEF00
• Other printing commands
!stackslots 0xABCDEF0
• U...
• Structure layout information in
the process memory
• DDR data generator generates
blob and code
• Detectable RAS structu...
Bjørn Vårdal IBM Runtime Technologies 33
DDR Components
OpenJ9Eclipse OMR
J9 JVM
DDR data
generator
DDR Interactive
RAS
Bjørn Vårdal IBM Runtime Technologies 34
RAS Structure
Runtime memory
VM structure
Threads
Port library
etc.
Bjørn Vårdal IBM Runtime Technologies 35
RAS Structure
Runtime memory
RAS Structure
Eye catcher: “J9VMRAS”
Bit pattern: 0x...
Bjørn Vårdal IBM Runtime Technologies 36
RAS Structure
Runtime memory
DDR blob
Structure layouts
Field offsets
Build flags...
Bjørn Vårdal IBM Runtime Technologies 37
DDR Structure Blob
Runtime memory
DDR blob
Structure layouts
Field offsets
Build ...
Bjørn Vårdal IBM Runtime Technologies 38
OMR ddrgen
Compile
with debug
symbols
ddrgen
blob.dat
Load into
memory at
runtime...
Bjørn Vårdal IBM Runtime Technologies 39
DDR Class Generation
superset.out
DDR Interactive
(j9ddr.jar)
Generate Java
struc...
Bjørn Vårdal IBM Runtime Technologies 40
DDR Interactive startup
Generated
Java classes
DDR Interactive
DDR blob
from core...
• DDR capabilities – Analyzing the JVM in a core file
• RAS structure – A detectable starting point
• OMR ddrgen – Creatin...
Bjørn Vårdal IBM Runtime Technologies 42
DDR in a new Runtime
• Small tutorial runtime with ~10 opcodes
• For illustrating basic OMR JITBuilder functionality
• For demonstrating basic ...
• Quick prototype
• ~1 hour of hacking
• Mostly copied from J9 JVM
Bjørn Vårdal IBM Runtime Technologies 44
DDR in Base9
Bjørn Vårdal IBM Runtime Technologies 45
RAS Structure in J9
JVM runtime memory
DDR blob
Structure layouts
Build flags
RAS...
Bjørn Vårdal IBM Runtime Technologies 46
RAS Structure in Base9
Base9 runtime memory
DDR blob
Structure layouts
Build flag...
Bjørn Vårdal IBM Runtime Technologies 47
Base9 Code Changes
Bjørn Vårdal IBM Runtime Technologies 48
Base9 Code Changes
Bjørn Vårdal IBM Runtime Technologies 49
Base9 Code Changes
Bjørn Vårdal IBM Runtime Technologies 50
RAS Structure in Base9
Base9 runtime memory
DDR blob
Structure layouts
Build flag...
Bjørn Vårdal IBM Runtime Technologies 51
DDR Interactive
Some changes were required:
• Disabled all J9 specific functional...
$
Bjørn Vårdal IBM Runtimes 52
DDR Interactive
Bjørn Vårdal IBM Runtime Technologies 53
What’s next?
• Make DDR Interactive plug-and-play
• Always support structure comm...
Bjørn Vårdal IBM Runtime Technologies 54
Conclusion
• DDR in J9
• Capabilities
• How DDR is implemented
• OMR ddrgen
• Tur...
Bjørn Vårdal IBM Runtime Technologies 55
Q & A
Nächste SlideShare
Wird geladen in …5
×

Secrets of building a debuggable runtime: Learn how language implementors solve your runtime issues

350 Aufrufe

Veröffentlicht am

Bjørn Vårdal, J9VM Software Developer, IBM, @bvaardal

New language runtimes appear all the time, but most of them die young. Failure can be attributed to
different reasons, but an important factor is that lack of support can limit the community’s and
industry’s willingness to adopt the new language.
Quicker development and improved serviceability allows emerging languages to overcome this obstacle.
By building on the proven technology available in Eclipse OMR, language developers can get more than
performance and stability; you also get tools that help you quickly debug your language runtime,
allowing you to provide competitive serviceability.
From this presentation, you will learn how to enable Eclipse OMR’s mature debugging features in your
language runtime, and also how Eclipse OMR can assist with development and debugging

Veröffentlicht in: Technologie
  • Gehören Sie zu den Ersten, denen das gefällt!

Secrets of building a debuggable runtime: Learn how language implementors solve your runtime issues

  1. 1. J9, OMR and beyond Serviceable Runtimes
  2. 2. • Bjørn Vårdal • IBM Runtimes Developer • J9 JVM • @bvaardal • MethodHandles • Lambdas • VarHandles • Project Valhalla Bjørn Vårdal IBM Runtime Technologies 2 Introduction
  3. 3. Bjørn Vårdal IBM Runtime Technologies 3 Most die young_______runtimes unless you service them.
  4. 4. The information contained in this presentation is provided for informational purposes only. Whilst efforts were made to verify the completeness and accuracy of the information contained in this presentation, it is provided “as is”, without warranty of any kind, express or implied. All performance data included in this presentation have been gathered in a controlled environment. Your own test results may vary based on hardware, software or infrastructure differences. All data included in this presentation are meant to be used only as a guide. In addition, the information contained in this presentation is based on IBM’s current product plans and strategy, which are subject to change by IBM, without notice. IBM and its affiliated companies shall not be responsible for any damages arising out of the use of, or otherwise related to, this presentation or any other documentation. Nothing contained in this presentation is intended to, or shall have the effect of: - creating any warrant or representation from IBM, its affiliated companies or its or their suppliers and/or licensors Bjørn Vårdal IBM Runtime Technologies 4 Disclaimer
  5. 5. • Advanced core dump analysis for J9 • Used by the J9 service team • Used by J9 developers • Front-end: DDR Interactive Bjørn Vårdal IBM Runtime Technologies 5 Direct Dump Reader
  6. 6. $ java -cp j9ddr.jar com.ibm.j9ddr.tools.ddrinteractive.DDRInteractive core.12345 Bjørn Vårdal IBM Runtime Technologies 6 DDR Interactive Ships with J9
  7. 7. $ java -cp j9ddr.jar com.ibm.j9ddr.tools.ddrinteractive.DDRInteractive core.12345 Bjørn Vårdal IBM Runtime Technologies 7 DDR Interactive
  8. 8. $ java -cp j9ddr.jar com.ibm.j9ddr.tools.ddrinteractive.DDRInteractive core.12345 *0 : PID: 2824; !j9javavm 0x7f27480eef0 Run !j9help to see a list of commands > Bjørn Vårdal IBM Runtime Technologies 8 DDR Interactive
  9. 9. $ java -cp j9ddr.jar com.ibm.j9ddr.tools.ddrinteractive.DDRInteractive core.12345 *0 : PID: 2824; !j9javavm 0x7f27480eef0 Run !j9help to see a list of commands > Bjørn Vårdal IBM Runtime Technologies 9 DDR Interactive
  10. 10. $ java -cp j9ddr.jar com.ibm.j9ddr.tools.ddrinteractive.DDRInteractive core.12345 *0 : PID: 2824; !j9javavm 0x7f27480eef0 Run !j9help to see a list of commands > !j9help Bjørn Vårdal IBM Runtime Technologies 10 DDR Interactive
  11. 11. $ java -cp j9ddr.jar com.ibm.j9ddr.tools.ddrinteractive.DDRInteractive core.12345 *0 : PID: 2824; !j9javavm 0x7f27480eef0 Run !j9help to see a list of commands > !j9help threads Lists VM threads stack Walks the Java stack for <thread> stackslots Walks the Java stack (including objects) for <thread> . . . classforname Find the class corresponding to name (with wildcards) dumpromclass Dump the specified J9ROMClass. Wild cards are allowed in class name. findpattern Search memory for a specific pattern whatis Determine the type of structure at a given address Bjørn Vårdal IBM Runtime Technologies 11 DDR - !j9help
  12. 12. $ java -cp j9ddr.jar com.ibm.j9ddr.tools.ddrinteractive.DDRInteractive core.20170315.010552.2824.01.dmp *0 : PID: 2824; !j9javavm 0x7f27480eef0 Run !j9help to see a list of commands > !threads Bjørn Vårdal IBM Runtime Technologies 12 DDR - !threads
  13. 13. $ java -cp j9ddr.jar com.ibm.j9ddr.tools.ddrinteractive.DDRInteractive core.20170315.010552.2824.01.dmp *0 : PID: 2824; !j9javavm 0x7f27480eef0 Run !j9help to see a list of commands > !threads !stack 0x0228d70 !j9vmthread 0x0228d70 !j9thread 0x07f274807d70 tid 0x4b (75) // (main) !stack 0x02294f0 !j9vmthread 0x02294f0 !j9thread 0x7f27480b0bf8 tid 0x4e (78) // (JIT Compilation Thread-1) !stack 0x022af20 !j9vmthread 0x022af20 !j9thread 0x7f27480bfcb0 tid 0x55 (85) // (JIT-SamplerThread) !stack 0x022b2e0 !j9vmthread 0x022b2e0 !j9thread 0x7f27480c0228 tid 0x56 (86) // (IProfiler) !stack 0x0238620 !j9vmthread 0x0238620 !j9thread 0x7f2748238170 tid 0x57 (87) // (Common-Cleaner) !stack 0x023f480 !j9vmthread 0x023f480 !j9thread 0x7f27482386e8 tid 0x58 (88) // (Signal Dispatcher) !stack 0x023f840 !j9vmthread 0x023f840 !j9thread 0x7f27483d01f8 tid 0x5a (90) // (Concurrent Mark Helper) !stack 0x0023fc0 !j9vmthread 0x0023fc0 !j9thread 0x7f27483d0c90 tid 0x5b (91) // (GC Slave) !stack 0x0002420 !j9vmthread 0x0002420 !j9thread 0x007f26e084d0 tid 0x5f (95) // (Attach API wait loop) !stack 0x0240730 !j9vmthread 0x0240730 !j9thread 0x7f27483d2218 tid 0x60 (96) // (Finalizer thread) > Bjørn Vårdal IBM Runtime Technologies 13 DDR - !threads
  14. 14. $ java -cp j9ddr.jar com.ibm.j9ddr.tools.ddrinteractive.DDRInteractive core.20170315.010552.2824.01.dmp *0 : PID: 2824; !j9javavm 0x7f27480eef0 Run !j9help to see a list of commands > !threads !stack 0x0228d70 !j9vmthread 0x0228d70 !j9thread 0x07f274807d70 tid 0x4b (75) // (main) !stack 0x02294f0 !j9vmthread 0x02294f0 !j9thread 0x7f27480b0bf8 tid 0x4e (78) // (JIT Compilation Thread-1) !stack 0x022af20 !j9vmthread 0x022af20 !j9thread 0x7f27480bfcb0 tid 0x55 (85) // (JIT-SamplerThread) !stack 0x022b2e0 !j9vmthread 0x022b2e0 !j9thread 0x7f27480c0228 tid 0x56 (86) // (IProfiler) !stack 0x0238620 !j9vmthread 0x0238620 !j9thread 0x7f2748238170 tid 0x57 (87) // (Common-Cleaner) !stack 0x023f480 !j9vmthread 0x023f480 !j9thread 0x7f27482386e8 tid 0x58 (88) // (Signal Dispatcher) !stack 0x023f840 !j9vmthread 0x023f840 !j9thread 0x7f27483d01f8 tid 0x5a (90) // (Concurrent Mark Helper) !stack 0x0023fc0 !j9vmthread 0x0023fc0 !j9thread 0x7f27483d0c90 tid 0x5b (91) // (GC Slave) !stack 0x0002420 !j9vmthread 0x0002420 !j9thread 0x007f26e084d0 tid 0x5f (95) // (Attach API wait loop) !stack 0x0240730 !j9vmthread 0x0240730 !j9thread 0x7f27483d2218 tid 0x60 (96) // (Finalizer thread) > Bjørn Vårdal IBM Runtime Technologies 14 DDR - !threads
  15. 15. $ java -cp j9ddr.jar com.ibm.j9ddr.tools.ddrinteractive.DDRInteractive core.20170315.010552.2824.01.dmp *0 : PID: 2824; !j9javavm 0x7f27480eef0 Run !j9help to see a list of commands > !threads !stack 0x0228d70 !j9vmthread 0x0228d70 !j9thread 0x07f274807d70 tid 0x4b (75) // (main) !stack 0x02294f0 !j9vmthread 0x02294f0 !j9thread 0x7f27480b0bf8 tid 0x4e (78) // (JIT Compilation Thread-1) !stack 0x022af20 !j9vmthread 0x022af20 !j9thread 0x7f27480bfcb0 tid 0x55 (85) // (JIT-SamplerThread) !stack 0x022b2e0 !j9vmthread 0x022b2e0 !j9thread 0x7f27480c0228 tid 0x56 (86) // (IProfiler) !stack 0x0238620 !j9vmthread 0x0238620 !j9thread 0x7f2748238170 tid 0x57 (87) // (Common-Cleaner) !stack 0x023f480 !j9vmthread 0x023f480 !j9thread 0x7f27482386e8 tid 0x58 (88) // (Signal Dispatcher) !stack 0x023f840 !j9vmthread 0x023f840 !j9thread 0x7f27483d01f8 tid 0x5a (90) // (Concurrent Mark Helper) !stack 0x0023fc0 !j9vmthread 0x0023fc0 !j9thread 0x7f27483d0c90 tid 0x5b (91) // (GC Slave) !stack 0x0002420 !j9vmthread 0x0002420 !j9thread 0x007f26e084d0 tid 0x5f (95) // (Attach API wait loop) !stack 0x0240730 !j9vmthread 0x0240730 !j9thread 0x7f27483d2218 tid 0x60 (96) // (Finalizer thread) > Bjørn Vårdal IBM Runtime Technologies 15 DDR - !threads
  16. 16. > !stack 0x0228d70 <1a2970> Generic special frame <1a2970> !j9method 0x01B9A718 X.print(Ljava/lang/String;)V <1a2970> !j9method 0x01B9A6F8 X.m(I)V <1a2970> !j9method 0x01B9A6D8 X.main([Ljava/lang/String;)V <1a2970> JNI call-in frame <1a2970> Native method frame > Bjørn Vårdal IBM Runtime Technologies 16 DDR - !stack
  17. 17. > !stack 0x0228d70 <1a2970> Generic special frame <1a2970> !j9method 0x01B9A718 X.print(Ljava/lang/String;)V <1a2970> !j9method 0x01B9A6F8 X.m(I)V <1a2970> !j9method 0x01B9A6D8 X.main([Ljava/lang/String;)V <1a2970> JNI call-in frame <1a2970> Native method frame > Bjørn Vårdal IBM Runtime Technologies 17 DDR - !stack
  18. 18. > !stackslots 0x0228d70 <1a2970> *** BEGIN STACK WALK, flags = 0401 walkThread = 0x01A2970 *** <1a2970> ITERATE_O_SLOTS <1a2970> RECORD_BYTECODE_PC_OFFSET <1a2970> Initial values: walkSP = 0x01AE6D60, PC = 0x01, literals = 0x0, A0 = 0x01AE6D78, j2iFrame = 0x0, ... <1a2970> Generic special frame: bp = 0x01AE6D78, sp = 0x01AE6D60, pc = 0x01, cp = 0x0, arg0EA = 0x01AE ... <1a2970> Bytecode frame: bp = 0x01AE6D90, sp = 0x01AE6D80, pc = 0x07FDCB43A6344, cp = 0x01B9A580, arg0EA = 0x01AE6D98, ... <1a2970> Method: X.print(Ljava/lang/String;)V !j9method 0x01B9A718 <1a2970> Bytecode index = 16 <1a2970> Using local mapper <1a2970> Locals starting at 0x01AE6D98 for 0x01 slots <1a2970> I-Slot: a0[0x01AE6D98] = 0x0FFEF46D0 <1a2970> Bytecode frame: bp = 0x01AE6DB0, sp = 0x01AE6DA0, pc = 0x07FDCB43A630A, cp = 0x01B9A580, arg0EA = 0x01AE6DB8, ... <1a2970> Method: X.m(I)V !j9method 0x01B9A6F8 <1a2970> Bytecode index = 6 <1a2970> Using local mapper <1a2970> Locals starting at 0x01AE6DB8 for 0x01 slots <1a2970> I-Slot: a0[0x01AE6DB8] = 0x01234 <1a2970> Bytecode frame: bp = 0x01AE6DD0, sp = 0x01AE6DC0, pc = 0x07FDCB43A62DB, cp = 0x1B9A580, arg0EA = 0x1AE6DD8, ... <1a2970> Method: X.main([Ljava/lang/String;)V !j9method 0x01B9A6D8 <1a2970> Bytecode index = 3 <1a2970> Using local mapper <1a2970> Locals starting at 0x01AE6DD8 for 0x01 slots <1a2970> I-Slot: a0[0x01AE6DD8] = 0x0FFEE5668 <1a2970> JNI call-in frame: bp = 0x01AE6E0, sp = 0x01AE6DE0, pc = 0x07FDCB96DE314, cp = 0x0, arg0EA = 0x01AE6E0, ... <1a2970> New ELS = 0x0 <1a2970> JNI native method frame: bp = 0x01AE6E88, sp = 0x01AE6E08, pc = 0x07, cp = 0x0, arg0EA = 0x01AE6E88, ... <1a2970> Object pushes starting at 0x01AE6E08 for 12 slots Bjørn Vårdal IBM Runtime Technologies 18 DDR - !stackslots
  19. 19. > !stackslots 0x0228d70 <1a2970> *** BEGIN STACK WALK, flags = 0401 walkThread = 0x01A2970 *** <1a2970> ITERATE_O_SLOTS <1a2970> RECORD_BYTECODE_PC_OFFSET <1a2970> Initial values: walkSP = 0x01AE6D60, PC = 0x01, literals = 0x0, A0 = 0x01AE6D78, j2iFrame = 0x0, ... <1a2970> Generic special frame: bp = 0x01AE6D78, sp = 0x01AE6D60, pc = 0x01, cp = 0x0, arg0EA = 0x01AE ... <1a2970> Bytecode frame: bp = 0x01AE6D90, sp = 0x01AE6D80, pc = 0x07FDCB43A6344, cp = 0x01B9A580, arg0EA = 0x01AE6D98, ... <1a2970> Method: X.print(Ljava/lang/String;)V !j9method 0x01B9A718 <1a2970> Bytecode index = 16 <1a2970> Using local mapper <1a2970> Locals starting at 0x01AE6D98 for 0x01 slots <1a2970> I-Slot: a0[0x01AE6D98] = 0x0FFEF46D0 <1a2970> Bytecode frame: bp = 0x01AE6DB0, sp = 0x01AE6DA0, pc = 0x07FDCB43A630A, cp = 0x01B9A580, arg0EA = 0x01AE6DB8, ... <1a2970> Method: X.m(I)V !j9method 0x01B9A6F8 <1a2970> Bytecode index = 6 <1a2970> Using local mapper <1a2970> Locals starting at 0x01AE6DB8 for 0x01 slots <1a2970> I-Slot: a0[0x01AE6DB8] = 0x01234 <1a2970> Bytecode frame: bp = 0x01AE6DD0, sp = 0x01AE6DC0, pc = 0x07FDCB43A62DB, cp = 0x1B9A580, arg0EA = 0x1AE6DD8, ... <1a2970> Method: X.main([Ljava/lang/String;)V !j9method 0x01B9A6D8 <1a2970> Bytecode index = 3 <1a2970> Using local mapper <1a2970> Locals starting at 0x01AE6DD8 for 0x01 slots <1a2970> I-Slot: a0[0x01AE6DD8] = 0x0FFEE5668 <1a2970> JNI call-in frame: bp = 0x01AE6E0, sp = 0x01AE6DE0, pc = 0x07FDCB96DE314, cp = 0x0, arg0EA = 0x01AE6E0, ... <1a2970> New ELS = 0x0 <1a2970> JNI native method frame: bp = 0x01AE6E88, sp = 0x01AE6E08, pc = 0x07, cp = 0x0, arg0EA = 0x01AE6E88, ... <1a2970> Object pushes starting at 0x01AE6E08 for 12 slots Bjørn Vårdal IBM Runtime Technologies 19 DDR - !stackslots
  20. 20. > !stackslots 0x0228d70 <1a2970> *** BEGIN STACK WALK, flags = 0401 walkThread = 0x01A2970 *** <1a2970> ITERATE_O_SLOTS <1a2970> RECORD_BYTECODE_PC_OFFSET <1a2970> Initial values: walkSP = 0x01AE6D60, PC = 0x01, literals = 0x0, A0 = 0x01AE6D78, j2iFrame = 0x0, ... <1a2970> Generic special frame: bp = 0x01AE6D78, sp = 0x01AE6D60, pc = 0x01, cp = 0x0, arg0EA = 0x01AE ... <1a2970> Bytecode frame: bp = 0x01AE6D90, sp = 0x01AE6D80, pc = 0x07FDCB43A6344, cp = 0x01B9A580, arg0EA = 0x01AE6D98, ... <1a2970> Method: X.print(Ljava/lang/String;)V !j9method 0x01B9A718 <1a2970> Bytecode index = 16 <1a2970> Using local mapper <1a2970> Locals starting at 0x01AE6D98 for 0x01 slots <1a2970> I-Slot: a0[0x01AE6D98] = 0x0FFEF46D0 <1a2970> Bytecode frame: bp = 0x01AE6DB0, sp = 0x01AE6DA0, pc = 0x07FDCB43A630A, cp = 0x01B9A580, arg0EA = 0x01AE6DB8, ... <1a2970> Method: X.m(I)V !j9method 0x01B9A6F8 <1a2970> Bytecode index = 6 <1a2970> Using local mapper <1a2970> Locals starting at 0x01AE6DB8 for 0x01 slots <1a2970> I-Slot: a0[0x01AE6DB8] = 0x01234 <1a2970> Bytecode frame: bp = 0x01AE6DD0, sp = 0x01AE6DC0, pc = 0x07FDCB43A62DB, cp = 0x1B9A580, arg0EA = 0x1AE6DD8, ... <1a2970> Method: X.main([Ljava/lang/String;)V !j9method 0x01B9A6D8 <1a2970> Bytecode index = 3 <1a2970> Using local mapper <1a2970> Locals starting at 0x01AE6DD8 for 0x01 slots <1a2970> I-Slot: a0[0x01AE6DD8] = 0x0FFEE5668 <1a2970> JNI call-in frame: bp = 0x01AE6E0, sp = 0x01AE6DE0, pc = 0x07FDCB96DE314, cp = 0x0, arg0EA = 0x01AE6E0, ... <1a2970> New ELS = 0x0 <1a2970> JNI native method frame: bp = 0x01AE6E88, sp = 0x01AE6E08, pc = 0x07, cp = 0x0, arg0EA = 0x01AE6E88, ... <1a2970> Object pushes starting at 0x01AE6E08 for 12 slots Bjørn Vårdal IBM Runtime Technologies 20 DDR - !stackslots
  21. 21. > !stackslots 0x0228d70 <1a2970> *** BEGIN STACK WALK, flags = 0401 walkThread = 0x01A2970 *** <1a2970> ITERATE_O_SLOTS <1a2970> RECORD_BYTECODE_PC_OFFSET <1a2970> Initial values: walkSP = 0x01AE6D60, PC = 0x01, literals = 0x0, A0 = 0x01AE6D78, j2iFrame = 0x0, ... <1a2970> Generic special frame: bp = 0x01AE6D78, sp = 0x01AE6D60, pc = 0x01, cp = 0x0, arg0EA = 0x01AE ... <1a2970> Bytecode frame: bp = 0x01AE6D90, sp = 0x01AE6D80, pc = 0x07FDCB43A6344, cp = 0x01B9A580, arg0EA = 0x01AE6D98, ... <1a2970> Method: X.print(Ljava/lang/String;)V !j9method 0x01B9A718 <1a2970> Bytecode index = 16 <1a2970> Using local mapper <1a2970> Locals starting at 0x01AE6D98 for 0x01 slots <1a2970> I-Slot: a0[0x01AE6D98] = 0x0FFEF46D0 <1a2970> Bytecode frame: bp = 0x01AE6DB0, sp = 0x01AE6DA0, pc = 0x07FDCB43A630A, cp = 0x01B9A580, arg0EA = 0x01AE6DB8, ... <1a2970> Method: X.m(I)V !j9method 0x01B9A6F8 <1a2970> Bytecode index = 6 <1a2970> Using local mapper <1a2970> Locals starting at 0x01AE6DB8 for 0x01 slots <1a2970> I-Slot: a0[0x01AE6DB8] = 0x01234 <1a2970> Bytecode frame: bp = 0x01AE6DD0, sp = 0x01AE6DC0, pc = 0x07FDCB43A62DB, cp = 0x1B9A580, arg0EA = 0x1AE6DD8, ... <1a2970> Method: X.main([Ljava/lang/String;)V !j9method 0x01B9A6D8 <1a2970> Bytecode index = 3 <1a2970> Using local mapper <1a2970> Locals starting at 0x01AE6DD8 for 0x01 slots <1a2970> I-Slot: a0[0x01AE6DD8] = 0x0FFEE5668 <1a2970> JNI call-in frame: bp = 0x01AE6E0, sp = 0x01AE6DE0, pc = 0x07FDCB96DE314, cp = 0x0, arg0EA = 0x01AE6E0, ... <1a2970> New ELS = 0x0 <1a2970> JNI native method frame: bp = 0x01AE6E88, sp = 0x01AE6E08, pc = 0x07, cp = 0x0, arg0EA = 0x01AE6E88, ... <1a2970> Object pushes starting at 0x01AE6E08 for 12 slots Bjørn Vårdal IBM Runtime Technologies 21 DDR - !stackslots
  22. 22. > !j9object 0x0FFEF46D0 J9VMJavaLangString at 0x00000000FFEF46D0 { struct J9Class* clazz = !j9class 0x1A5B500 // java/lang/String Object flags = 0x00000000; [B value = !fj9object 0x86f36828 (offset=0) (java/lang/String) I count = 0x00000007 (offset=4) (java/lang/String) I hashCode = 0x00000000 (offset=8) (java/lang/String) "foo4660" } Bjørn Vårdal IBM Runtime Technologies 22 DDR - !j9object
  23. 23. > !j9object 0x0FFEF46D0 J9VMJavaLangString at 0x00000000FFEF46D0 { struct J9Class* clazz = !j9class 0x1A5B500 // java/lang/String Object flags = 0x00000000; [B value = !fj9object 0x86f36828 (offset=0) (java/lang/String) I count = 0x00000007 (offset=4) (java/lang/String) I hashCode = 0x00000000 (offset=8) (java/lang/String) "foo4660" } Bjørn Vårdal IBM Runtime Technologies 23 DDR - !j9object
  24. 24. GDB • Requires intimate knowledge about the JVM internal structures • Requires debug symbols • Source must match • Platform specific DDR • Has intimate knowledge about the JVM internal structures • Doesn’t require debug symbols • Cross JVM version • Cross platform Bjørn Vårdal IBM Runtime Technologies 24 GDB vs DDR
  25. 25. > !stackslots 0x0228d70 <1a2970> *** BEGIN STACK WALK, flags = 0401 walkThread = 0x01A2970 *** <1a2970> ITERATE_O_SLOTS <1a2970> RECORD_BYTECODE_PC_OFFSET <1a2970> Initial values: walkSP = 0x01AE6D60, A0 = 0x01AE6D78, <1a2970> Generic special frame: bp = 0x01AE6D78, sp = 0x01AE6D60, <1a2970> Bytecode frame: bp = 0x01AE6D90, sp = 0x01AE6D80, arg0EA = <1a2970> Method: X.print(Ljava/lang/String;)V !j9method 0x01B9A718 <1a2970> Bytecode index = 16 <1a2970> Using local mapper <1a2970> Locals starting at 0x01AE6D98 for 0x01 slots <1a2970> I-Slot: a0[0x01AE6D98] = 0x0FFEF46D0 <1a2970> Bytecode frame: bp = 0x01AE6DB0, sp = 0x01AE6DA0, pc = <1a2970> Method: X.m(I)V !j9method 0x01B9A6F8 <1a2970> Bytecode index = 6 <1a2970> Using local mapper <1a2970> Locals starting at 0x01AE6DB8 for 0x01 slots <1a2970> I-Slot: a0[0x01AE6DB8] = 0x01234 <1a2970> Bytecode frame: bp = 0x01AE6DD0, arg0EA = 0x1AE6DD8 <1a2970> Method: X.main([Ljava/lang/String;)V !j9method 0x01B9A6D8 <1a2970> Bytecode index = 3 <1a2970> Using local mapper <1a2970> Locals starting at 0x01AE6DD8 for 0x01 slots <1a2970> I-Slot: a0[0x01AE6DD8] = 0x0FFEE5668 <1a2970> JNI call-in frame: sp = 0x01AE6DE0, pc = 0x07FDCB96DE314 <1a2970> JNI native method frame: cp = 0x0, arg0EA = 0x01AE6E88, ... <1a2970> Object pushes starting at 0x01AE6E08 for 12 slots (gdb) x/200 0x0000000001AE6D60 0x1ae6d60: 0x00000000 0x00000000 0x01b9a718 0x00000000 0x1ae6d70: 0xb43a6344 0x00007fdc 0x01ae6d9a 0x00000000 0x1ae6d80: 0x01b9a6f8 0x00000000 0xb43a630a 0x00007fdc 0x1ae6d90: 0x01ae6db8 0x00000000 0xffef46d0 0x00000000 0x1ae6da0: 0x01b9a6d8 0x00000000 0xb43a62db 0x00007fdc 0x1ae6db0: 0x01ae6dd8 0x00000000 0x00001234 0x00000000 0x1ae6dc0: 0x00000000 0x00000000 0xb96de314 0x00007fdc 0x1ae6dd0: 0x01ae6e00 0x00000000 0xffee5668 0x00000000 0x1ae6de0: 0x00000000 0x00000000 0x00020000 0x00000000 0x1ae6df0: 0x00000060 0x00000000 0x00000007 0x00000000 0x1ae6e00: 0x01ae6e8a 0x00000000 0xffee5668 0x00000000 0x1ae6e10: 0x86ef0828 0x00000000 0x86f317d0 0x00000000 0x1ae6e20: 0x86f317d0 0x00000000 0x00000000 0x00000000 0x1ae6e30: 0x00000000 0x00000000 0x86f2ed50 0x00000000 0x1ae6e40: 0x86ef3548 0x00000000 0x86ef34b8 0x00000000 0x1ae6e50: 0x86ef0240 0x00000000 0x86ef0fb0 0x00000000 DDR Bjørn Vårdal IBM Runtime Technologies 25 GDB vs DDR GDB
  26. 26. Bjørn Vårdal IBM Runtime Technologies 26 DDR Interactive ”The tool that I use the most and that saves me the most time.” J9 Service Team Lead “We can implement commands to look for known symptoms, which drastically reduces our triage work.” J9 Service Team Member
  27. 27. • We are open-sourcing the J9 JVM • DDR coming along for the ride  • http://openj9.org Bjørn Vårdal IBM Runtime Technologies 27 OpenJ9
  28. 28. Reliable Shared Components for High Performance Language Runtimes http://www.eclipse.org/omr https://github.com/eclipse/omr https://developer.ibm.com/open/omr Bjørn Vårdal IBM Runtime Technologies 28 Eclipse OMR
  29. 29. Bjørn Vårdal IBM Runtime Technologies 29 Agenda • DDR in J9 • Capabilities • How DDR is implemented • OMR ddrgen • Generating the DDR resources • DDR in a new runtime • Adding DDR capabilities to Base9 • Loading a Base9 core into DDR Interactive
  30. 30. • Print structure at address !j9object 0x12345678 !j9method 0xABCDEF00 • Other printing commands !stackslots 0xABCDEF0 • Utility functions !classforname !findpattern !whatis • Data analysis gccheck deadlock Bjørn Vårdal IBM Runtime Technologies 30 What can DDR do? >
  31. 31. • Print structure at address !j9object 0x12345678 !j9method 0xABCDEF00 • Other printing commands !stackslots 0xABCDEF0 • Utility functions !classforname !findpattern !whatis • Data analysis gccheck deadlock Bjørn Vårdal IBM Runtime Technologies 31 What can DDR do? > gccheck Starting GC Check Checking HEAP...done (919 ms). Checking CLASS HEAP...done (296 ms). Checking REMEMBERED SET...done (54 ms). Checking UNFINALIZED...done (3 ms). Checking FINALIZABLE...done (3 ms). Checking OWNABLE_SYNCHRONIZER...done (1 ms). Checking STRING TABLE...done (516 ms). Checking CLASS LOADERS...done (4 ms). Checking JNI GLOBAL REFS...done (1 ms). Checking JNI WEAK GLOBAL REFS...done (1 ms). Checking JVMTI OBJECT TAG TABLES...done (3 ms). Checking VM CLASS SLOTS...done (1 ms). Checking MONITOR TABLE...done (7 ms). Checking VM THREAD SLOTS...done (208 ms). Checking THREAD STACKS...done (25 ms). Done (2058ms)
  32. 32. • Structure layout information in the process memory • DDR data generator generates blob and code • Detectable RAS structure in memory at runtime Bjørn Vårdal IBM Runtime Technologies 32 How does DDR work? Runtime DDR InteractiveDDR data generator DDR blob Core dump DDR blob Generated code RAS RAS
  33. 33. Bjørn Vårdal IBM Runtime Technologies 33 DDR Components OpenJ9Eclipse OMR J9 JVM DDR data generator DDR Interactive RAS
  34. 34. Bjørn Vårdal IBM Runtime Technologies 34 RAS Structure Runtime memory VM structure Threads Port library etc.
  35. 35. Bjørn Vårdal IBM Runtime Technologies 35 RAS Structure Runtime memory RAS Structure Eye catcher: “J9VMRAS” Bit pattern: 0xAA55AA55 ... DDR Structure blob … VM … VM structure Threads Port library etc.
  36. 36. Bjørn Vårdal IBM Runtime Technologies 36 RAS Structure Runtime memory DDR blob Structure layouts Field offsets Build flags RAS Structure Eye catcher: “J9VMRAS” Bit pattern: 0xAA55AA55 ... DDR Structure blob … VM … VM structure Threads Port library etc.
  37. 37. Bjørn Vårdal IBM Runtime Technologies 37 DDR Structure Blob Runtime memory DDR blob Structure layouts Field offsets Build flags RAS Structure Eye catcher: “J9VMRAS” Bit pattern: 0xAA55AA55 ... DDR Structure blob … VM … VM structure Threads Port library etc.
  38. 38. Bjørn Vårdal IBM Runtime Technologies 38 OMR ddrgen Compile with debug symbols ddrgen blob.dat Load into memory at runtime Scan source files for build flags ./getMacros.sh . libdwarf
  39. 39. Bjørn Vårdal IBM Runtime Technologies 39 DDR Class Generation superset.out DDR Interactive (j9ddr.jar) Generate Java structure classes superset.out Generate Java pointer classes
  40. 40. Bjørn Vårdal IBM Runtime Technologies 40 DDR Interactive startup Generated Java classes DDR Interactive DDR blob from core file DDR ClassLoader Template Field offsets Load structure classes
  41. 41. • DDR capabilities – Analyzing the JVM in a core file • RAS structure – A detectable starting point • OMR ddrgen – Creating the DDR blob and superset • DDR Interactive – Using the blob and superset Bjørn Vårdal IBM Runtime Technologies 41 Summary
  42. 42. Bjørn Vårdal IBM Runtime Technologies 42 DDR in a new Runtime
  43. 43. • Small tutorial runtime with ~10 opcodes • For illustrating basic OMR JITBuilder functionality • For demonstrating basic DDR functionality? • My fork: https://github.com/bjornvar/Base9/tree/ddr Bjørn Vårdal IBM Runtime Technologies 43 Base9
  44. 44. • Quick prototype • ~1 hour of hacking • Mostly copied from J9 JVM Bjørn Vårdal IBM Runtime Technologies 44 DDR in Base9
  45. 45. Bjørn Vårdal IBM Runtime Technologies 45 RAS Structure in J9 JVM runtime memory DDR blob Structure layouts Build flags RAS Structure Eye catcher: “J9VMRAS” Bit pattern: 0xAA55AA55 ... DDR Structure blob … JavaVM … JavaVM structure Threads Port library etc.
  46. 46. Bjørn Vårdal IBM Runtime Technologies 46 RAS Structure in Base9 Base9 runtime memory DDR blob Structure layouts Build flags Generated by running ddrgen on Base9 RAS Structure Eye catcher: “J9VMRAS” Bit pattern: 0xAA55AA55 ... DDR Structure blob … ExecutionContext Copied the structure from the J9 source ExecutionContext Stack String table etc.
  47. 47. Bjørn Vårdal IBM Runtime Technologies 47 Base9 Code Changes
  48. 48. Bjørn Vårdal IBM Runtime Technologies 48 Base9 Code Changes
  49. 49. Bjørn Vårdal IBM Runtime Technologies 49 Base9 Code Changes
  50. 50. Bjørn Vårdal IBM Runtime Technologies 50 RAS Structure in Base9 Base9 runtime memory DDR blob Structure layouts Build flags RAS Structure Eye catcher: “J9VMRAS” Bit pattern: 0xAA55AA55 ... DDR Structure blob … ExecutionContext ExecutionContext Stack String table etc.
  51. 51. Bjørn Vårdal IBM Runtime Technologies 51 DDR Interactive Some changes were required: • Disabled all J9 specific functionality (!stackslots, etc.) • Told it to look for B9RAS instead of J9RAS • Added build flags constants
  52. 52. $ Bjørn Vårdal IBM Runtimes 52 DDR Interactive
  53. 53. Bjørn Vårdal IBM Runtime Technologies 53 What’s next? • Make DDR Interactive plug-and-play • Always support structure commands on DDR enabled core files • Easy DDR generation process • Release DDR Interactive and DDR generators • OMR • OpenJ9
  54. 54. Bjørn Vårdal IBM Runtime Technologies 54 Conclusion • DDR in J9 • Capabilities • How DDR is implemented • OMR ddrgen • Turning debug symbols into blob and superset • DDR in Base9 • Adding DDR capabilities to a new runtime • Loading a Base9 core into DDR Interactive
  55. 55. Bjørn Vårdal IBM Runtime Technologies 55 Q & A

×