This document discusses auditing reactive applications to detect blocking API calls. It describes how blocking calls can negatively impact performance in reactive systems by consuming thread pools. Various techniques for detecting blocking calls are examined, including modifying the JDK, generating warnings during compilation, and instrumenting code at runtime using a JVM agent. Aspect programming is highlighted as a way to audit applications at load time by weaving in checks for over 500 blocking methods across many Java APIs. The reactive-audit tool is introduced as an open source project for helping developers test for blocking calls in frameworks like Play, Jetty, and Akka.
1. Audit your reactive
applications
+PhilippePrados @pprados
+François-Xavier Bonnet @fxbonnet
2. What is reactive?
● Responsive
● Resilient
● Elastic
● Message driven
“Non-blocking communication allows recipients
to only consume resources while active,
leading to less system overhead.”
http://www.reactivemanifesto.org/
3. The problem: blocking APIs
● Blocking APIs consume the thread pool
because of too much waiting time
● We should have no more threads than cores
on the machine
4. How do I track blocking API calls?
● Modifying the JVM?
● Generating warnings during compilation?
● Instrumenting the code at runtime using a
JVM agent?
Code Compiler JRE
5. Modifying the JDK
● This strategy has been used by Google in
Android SDK in order to avoid the use of
blocking API in a UI thread
● Not too hard for pure java methods but what
about native methods (OS specific)?
6. Generating warnings during
compilation?
● Annotate JDK methods inside an IDE? Used
by IntelliJ for @Nullable and @NotNull
annotations but not standard
● JSR 269 allows to do things at compile time
https://jcp.org/en/jsr/detail?id=269
● Compilation occurs in several phases, we
can access the syntax tree only after the first
phase
o The pure syntax, not the called method name
7. Generating warnings during
compilation?
● JSR308 allows to do things after the second
phase http://types.cs.washington.edu/jsr308/
o The augmented syntax tree
● But not included in JDK 8
8. Generating warnings during
compilation?
● Many things can be seen only at runtime
● Ex: java.io.Reader.read() is blocking for a file
or network connection but not for memory
9. Instrumenting the code at runtime
using a JVM agent?
● Patch the code at application startup
● Solution:
10. Auditing using Aspect programing
● Load time weaving
● Java agent at JVM startup
● List all blocking methods
● But, can not be weaving the SDK itself
Some tweaks:
● Some methods might be blocking depending
on the arguments or internal use!
14. Log output with Play
HIGH : Call method void java.io.PrintWriter.print(String) with /tmp/sbt1269029892078438540.log
com.octo.reactive.audit.lib.FileReactiveAuditException:
Call method void java.io.PrintWriter.print(String) with /tmp/sbt1269029892078438540.log
at thread "play-internal-execution-context-1"
at sbt.ConsoleOut$$anon$3.print(ConsoleOut.scala:52)
at sbt.ConsoleLogger.setColor(ConsoleLogger.scala:158)
at sbt.ConsoleLogger.reset(ConsoleLogger.scala:153)
at sbt.ConsoleLogger.sbt$ConsoleLogger$$printLabeledLine(ConsoleLogger.scala:168)
at sbt.ConsoleLogger$$anonfun$log$1.apply(ConsoleLogger.scala:164)
at sbt.ConsoleLogger$$anonfun$log$1.apply(ConsoleLogger.scala:163)
at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)
at sbt.ConsoleLogger.log(ConsoleLogger.scala:163)
at sbt.ConsoleLogger.log(ConsoleLogger.scala:151)
at sbt.AbstractLogger.log(Logger.scala:31)
...
15. Use it with in testing scope
● Because the analysis is performed on
effective method calls at runtime
● The audit should be used while running
o Integration tests
o Functional tests
● Not relevant in unit tests
16. Configuration
● Alert for all calls to blocking methods?
● We need some filters
o Startup shift
o Thread name (pattern matching)
o Kind and level of blocking call (CPU, File, Network)
o Annotations
21. Build integration and continuous
testing
● Sample script for
o Maven
o Gradle
o Sbt
● You’re welcome to propose others
22. Use it with your reactive project
● Play
● Akka
● VertX
● AsyncHttpClient
● RxJava
● …
23. We need you !
● Check it with your project
● Improve the rules
● Submit some extensions
● Submit plugins for others tools
● Submit XSTL to format XML log file
https://github.com/octo-online/reactive-audit