Consuming web services asynchronously with Futures and Rx Observables (svcc, svcc2013)

@crichardson
Consuming web services
asynchronously with Futures
and Rx Observables
Chris Richardson
Author of POJOs in Action
Founder of the original CloudFoundry.com
@crichardson
chris@chrisrichardson.net
http://plainoldobjects.com

@crichardson
Presentation goal
Learn how to use (Scala)
Futures and Rx
Observables to write simple
yet robust and scalable
concurrent code

@crichardson
About Chris
http://www.theregister.co.uk/2009/08/19/springsource_cloud_foundry/

@crichardson
Agenda
The need for concurrency
Simplifying concurrent code with Futures
Taming callback hell with JavaScript promises
Consuming asynchronous streams with Reactive
Extensions

@crichardson
Let’s imagine you are building
an online store

@crichardson
Shipping
Recomendations
Product
Info
Reviews

@crichardson
Reviews
Product
Info Sales
ranking

@crichardson
Related
books
Viewing
history
Forum

@crichardson
Application architecture
Product Info Service
Desktop
browser
Native
mobile
client
REST
REST
REST
Recomendation
Service
Review Service
Front end server
API gatewayREST
Mobile
browser
Web Application
HTML/
JSON
JSON

@crichardson
Handling getProductDetails()
Front-end
server
Recommendations
Service
Review
Service
getProductInfo()
getRecommendations()
getReviews()
getProductDetails()

@crichardson
Handling getProductDetails() -
sequentially
Front-end
server
Recommendations
Service
Review
Service
getProductInfo()
getReviews()
getProductDetails()
Higher response time :-(

@crichardson
Handling getProductDetails() -
in parallel
Front-end
server
Recommendations
Service
Review
Service
getProductInfo()
getReviews()
getProductDetails()
Lower response
time :-)

@crichardson
Implementing a concurrent
REST client
Thread-pool based approach
executorService.submit(new Callable(...))
Simpler but less scalable - lots of idle threads
consuming memory
Event-driven approach
NIO with completion callbacks
More complex but more scalable

@crichardson
The front-end server must handle
partial failure of backend services
Front-end
server
Recommendations
Service
Review
Service
getProductInfo()
getReviews()
getProductDetails()
X
How to provide
a good user
experience?

@crichardson
Futures are a great
concurrency abstraction
http://en.wikipedia.org/wiki/Futures_and_promises

@crichardson
Worker threadMain thread
How futures work
Outcome
Future
Client
get
Asynchronous
operation
set
initiates

@crichardson
Beneﬁts
Client does not know how the asynchronous operation is
implemented
Client can invoke multiple asynchronous operations and
gets a Future for each one.

@crichardson
Handling ProductDetails request
ProductDetailsController
ProductDetailsService
getProductDetails()
ProductInfoService
getProductInfo()
ReviewService
getReviews()
RecommendationService
RestTemplate

@crichardson
REST client using Spring @Async
@Component
class ProductInfoServiceImpl extends ProducInfoService {
val restTemplate : RestTemplate = ...
@Async
def getProductInfo(productId: Long) = {
new AsyncResult(restTemplate.getForObject(....)...)
}
}
Execute
asynchronously in
thread pool
A fulﬁlled Future
trait ProductInfoService {
def getProductInfo(productId: Long):
java.util.concurrent.Future[ProductInfo]
}

@crichardson
ProductDetailsService
@Component
class ProductDetailsService
@Autowired()(productInfoService: ProductInfoService,
reviewService: ReviewService,
recommendationService: RecommendationService) {
def getProductDetails(productId: Long): ProductDetails = {
val productInfoFuture = productInfoService.getProductInfo(productId)
}
}
val recommendationsFuture = recommendationService.getRecommendations(pr
val reviewsFuture = reviewService.getReviews(productId)
val productInfo = productInfoFuture.get(300, TimeUnit.MILLISECONDS)
val recommendations =
recommendationsFuture.get(10, TimeUnit.MILLISECONDS)
val reviews = reviewsFuture.get(10, TimeUnit.MILLISECONDS)
ProductDetails(productInfo, recommendations, reviews)

@crichardson
ProductController
@Controller
class ProductController
@Autowired()(productDetailsService : ProductDetailsService)
{
@RequestMapping(Array("/productdetails/{productId}"))
@ResponseBody
def productDetails(@PathVariable productId: Long) =
productDetailsService.getProductDetails(productId)

@crichardson
Blocks Tomcat thread until
Future completes
Not bad but...
def getProductDetails(productId: Long): ProductDetails = {
val productInfo =
productInfoFuture.get(300, TimeUnit.MILLISECONDS)
Not so scalable :-(

@crichardson
... and also...
Java Futures work well for a single-level of asynchronous
execution
BUT
#fail for more complex, scalable scenarios
Difficult to compose and coordinate multiple concurrent
operations
See this blog post for more details:
http://techblog.netflix.com/2013/02/rxjava-netflix-api.html

@crichardson
Better: Futures with callbacks
no blocking!
val f : Future[Int] = Future { ... }
f onSuccess {
case x : Int => println(x)
}
f onFailure {
case e : Exception =>
println("exception thrown")
}
Guava ListenableFutures, Spring 4 ListenableFuture
Java 8 CompletableFuture, Scala Futures

@crichardson
Even better: composable futures
def asyncOperation() = Future { ... ; 1 }
val r = asyncOperation().map{ _ * 2 }
assertEquals(4, Await.result(r, 1 second))
Scala, Java 8 CompletableFuture (partially)
Transforms Future

@crichardson
map() is asynchronous - uses callbacks
class Future[T] {
def map[S](f: T => S): Future[S] = {
val p = Promise[S]()
onSuccess {
case r => p success f(r)
}
onFailure {
case t => p failure t
}
p.future
}
When ‘this’ completes
propagate outcome to ‘p’
Return new Future

@crichardson
Chaining using ﬂatmap()
def asyncOperation() = Future { ... ; 3 }
def asyncSquare(x : Int) : Future[Int] = Future { x * x}
val r = asyncOperation().flatMap { x => asyncSquare(x) }
assertEquals(9, Await.result(r, 1 second))
Chaining asynchronous
operations

@crichardson
Combining futures
val f1 = Future { ... ; 1}
val f2 = Future { .... ; 2}
val fzip = f1 zip f2
assertEquals((1, 2), Await.result(fzip, 1 second))
def asyncOp(x : Int) = Future { x * x}
val f = Future.sequence((1 to 5).map { x => asyncOp(x) })
assertEquals(List(1, 4, 9, 16, 25),
Await.result(f, 1 second))
Combines two futures
Transforms list of futures to a
future containing a list

@crichardson
Scala futures are Monads
def callB() : Future[...] = ...
def callC() : Future[...] = ...
def callD() : Future[...] = ...
val result = for {
(b, c) <- callB() zip callC();
d <- callD(b, c)
} yield d
result onSuccess { .... }
Two calls execute in parallel
And then invokes D
Get the result of D

@crichardson
‘for’ is shorthand for map() and ﬂatMap()
(callB() zip callC()) flatMap { r1 =>
val (b, c) = r1
callD(b, c) map { d => d }
}
for {
(b, c) <- callB() zip callC();
d <- callD(b, c)
} yield d

@crichardson
Scala Future + RestTemplate
import scala.concurrent.Future
@Component
class ProductInfoService {
def getProductInfo(productId: Long): Future[ProductInfo]
= {
Future { restTemplate.getForObject(....) }
}
}
Executes in thread pool
Scala Future

@crichardson
Scala Future + RestTemplate
class ProductDetailsService @Autowired()(....) {
def getProductDetails(productId: Long) = {
val productInfoFuture = productInfoService.getProductInfo(productId)
val recommendationsFuture =
recommendationService.getRecommendations(productId)
val reviewsFuture = reviewService.getReviews(productId)
for (((productInfo, recommendations), reviews) <-
productInfoFuture zip recommendationsFuture zip reviewsFuture)
yield ProductDetails(productInfo, recommendations, reviews)
}
}
Non-blocking!

@crichardson
Async Spring MVC + Scala
Futures
@Controller
class ProductController ... {
@ResponseBody
def productDetails(@PathVariable productId: Long) = {
val productDetails =
val result = new DeferredResult[ProductDetails]
productDetails onSuccess {
case r => result.setResult(r)
}
productDetails onFailure {
case t => result.setErrorResult(t)
}
result
}
Convert Scala
Future to
Spring MVC
DeferredResult

@crichardson
Servlet layer is asynchronous
but the backend uses thread
pools
Need event-driven REST
client

@crichardson
New in Spring 4
Mirrors RestTemplate
Methods return a ListenableFuture
ListenableFuture = JDK 7 Future + callback methods
Can use HttpComponents NIO-based AsyncHttpClient
Spring AsyncRestTemplate

@crichardson
Using the AsyncRestTemplate
http://hc.apache.org/httpcomponents-asyncclient-dev/
val asyncRestTemplate = new AsyncRestTemplate(
new HttpComponentsAsyncClientHttpRequestFactory())
override def getProductInfo(productId: Long) = {
val listenableFuture =
asyncRestTemplate.getForEntity("{baseUrl}/productinfo/{productId}",
classOf[ProductInfo],
baseUrl, productId)
toScalaFuture(listenableFuture).map { _.getBody }
}
Convert to Scala Future and get entity

@crichardson
Converting ListenableFuture to
Scala Future
implicit def toScalaFuture[T](f :
ListenableFuture[T]) : Future[T] = {
val p = promise[T]()
f.addCallback(new ListenableFutureCallback[T] {
def onSuccess(result: T) { p.success(result)}
def onFailure(t: Throwable) { p.failure(t) }
})
p.future
}
The producer side of Scala Futures
Supply outcome
Return future

@crichardson
Now everything is non-
blocking :-)

@crichardson
If recommendation service is
down...
Never responds front-end server waits indeﬁnitely
Consumes valuable front-end server resources
Page never displayed and customer gives up
Returns an error
Error returned to the front-end server error page is displayed
Customer gives up

@crichardson
Fault tolerance at Netflix
Network timeouts and retries
Invoke remote services via a bounded thread pool
Use the Circuit Breaker pattern
On failure:
return default/cached data
return error to caller
Implementation: https://github.com/Netflix/Hystrix
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-
volume.html

@crichardson
How to handling partial
failures?
productDetailsFuture zip
recommendationsFuture zip
reviewsFuture
Fails if any Future
has failed

@crichardson
Handling partial failures
recommendationService.
getRecommendations(userId,productId)
.recover {
case _ => Recommendations(List())
}
“catch-like” Maps Throwable to
value

@crichardson
Why solve this problem for
JavaScript?
Browser invokes web services
Implement front-end server/API gateway using NodeJS

@crichardson
What’s NodeJS?
Designed for DIRTy apps

@crichardson
NodeJS
JavaScript
Reactor
pattern
Modules

@crichardson
Asynchronous JavaScript
code = callback hell
Scenarios:
Sequential: A B C
Fork and join: A and B C
Code quickly becomes very messy

@crichardson
Callback-based HTTP client
request = require("request")
handler = (error, clientResponse, body) ->
if clientResponse.statusCode != 200
// ERROR
else
// SUCCESS
request("http://.../productdetails/" + productId,
handler)

@crichardson
Messy callback code
getProductDetails = (req, res) ->
productId = req.params.productId
result = {productId: productId}
makeHandler = (key) ->
(error, clientResponse, body) ->
if clientResponse.statusCode != 200
res.status(clientResponse.statusCode)
res.write(body)
res.end()
else
result[key] = JSON.parse(body)
if (result.productInfo and result.recommendations and result.reviews)
res.json(result)
request("http://localhost:3000/productdetails/" + productId, makeHandler("productInfo"))
request("http://localhost:3000/recommendations/" + productId,
makeHandler('recommendations'))
request("http://localhost:3000/reviews/" + productId, makeHandler('reviews'))
app.get('/productinfo/:productId', getProductInfo)
Returns a callback
function
Have all three requests
completed?

@crichardson
Simplifying code with
Promises (a.k.a. Futures)
Functions return a promise - no callback parameter
A promise represents an eventual outcome
Use a library of functions for transforming and composing
promises
Promises/A+ speciﬁcation - http://promises-aplus.github.io/promises-spec

@crichardson
Basic promise API
promise2 = promise1.then(fulfilledHandler, errorHandler, ...)
called when promise1 is fulﬁlled
returns a promise or value
resolved with outcome of
callback
called when promise1 is
rejected

About when.js
Rich API = Promises spec
+ use full extensions
Creation:
when.defer()
when.reject()...
Combinators:
all, some, join, map,
reduce
Other:
Calling non-promise
code
timeout
....
https://github.com/cujojs/when

@crichardson
Simpler promise-based code
rest = require("rest")
@httpClient = rest.chain(mime).chain(errorCode);
getProductInfo : (productId) ->
@httpClient
path: "http://.../productinfo/" + productId
Returns a promise
No ugly callbacks

@crichardson
Simpler promise-based code
getProductDetails: (productId) ->
makeProductDetails =
(productInfo, recommendations, reviews) ->
details =
productId: productId
productDetails: productInfo.entity
recommendations: recommendations.entity
reviews: reviews.entity
details
responses = [@getProductInfo(productId),
@getRecommendations(productId),
@getReviews(productId)]
all(responses).spread(makeProductDetails)
Array[Promise] Promise[Array]

@crichardson
Simpler promise-based code:
HTTP handler
getProductDetails = (req, res) ->
productId = req.params.productId
succeed = (productDetails) -> res.json(productDetails)
fail = (something) ->
res.send(500, JSON.stringify(something.entity ||
something))
productDetailsService.
getDetails(productId).
then(succeed, fail)
app.get('/productdetails/:productId', getProductDetails)

@crichardson
Writing robust client code

@crichardson
Recovering from failures
getRecommendations(productId)
.otherwise( -> {})
Invoked to return default value if
getRecommendations() fails

@crichardson
Let’s imagine you have a
stream of trades
and
you need to calculate the rolling
average price of each stock

@crichardson
Spring Integration + Complex
event processing (CEP)
engine is a good choice

@crichardson
But where is the high-level
abstraction that solves this
problem?

@crichardson
Future[List[T]]
Not applicable to inﬁnite
streams

@crichardson
Introducing Reactive
Extensions (Rx)
The Reactive Extensions (Rx) is a library
for composing asynchronous and event-
based programs using observable
sequences and LINQ-style query
operators. Using Rx, developers
represent asynchronous data streams
with Observables , query
asynchronous data streams using
LINQ operators , and .....
https://rx.codeplex.com/

@crichardson
About RxJava
Reactive Extensions (Rx) for the JVM
Implemented in Java
Adaptors for Scala, Groovy and Clojure
https://github.com/Netﬂix/RxJava

@crichardson
RxJava core concepts
class Observable<T> {
Subscription subscribe(Observer<T> observer)
...
}
interface Observer<T> {
void onNext(T args)
void onCompleted()
void onError(Throwable e)
}
Notiﬁes
An asynchronous
stream of items
Used to
unsubscribe

@crichardson
Comparing Observable to...
Observer pattern - similar but adds
Observer.onComplete()
Observer.onError()
Iterator pattern - mirror image
Push rather than pull
Future - similar but
Represents a stream of multiple values

@crichardson
Transforming Observables
Observable<T> take(int n);
Observable<T> skip(int n);
<R> Observable<R> map(Func1<T,R> func)
<R> Observable<R> flatMap(Func1<T,Observable<R>> func)
Observable<T> filter(Func1<T,java.lang.Boolean> predicate)
...
}
Scala-collection style methods

@crichardson
Transforming observables
<K> Observable<GroupedObservable<K,T>>
groupBy(Func1<T,K> keySelector)
...
}
Similar to Scala groupBy except results are
emitted as items arrive
Stream of streams!

@crichardson
Combining observables
static <R,T1,T2>
Observable<R> zip(Observable<T0> o1,
Observable<T1> o2,
Func2<T1,T2,R> reduceFunction)
...
}
Invoked with pairs of
items from o1 and o2
Stream of results from
reduceFunction

@crichardson
Creating observables from
data
static Observable<T> from(Iterable<T> iterable]);
static Observable<T> from(T items ...]);
static Observable<T> from(Future<T> future]);
...
}

@crichardson
Arranges to call obs.onNext/
onComplete/...
Creating observables from
event sources
Observable<T> o = Observable.create(
new SubscriberFunc<T> () {
Subscription onSubscribe(Observer<T> obs) {
... connect to event source....
return new Subscriber () {
void unsubscribe() { ... };
};
});
Called
once for
each
Observer
Called when
unsubscribing

@crichardson
Using Rx Observables
instead of Futures

@crichardson
Rx-based ProductInfoService
@Component
class ProductInfoService
override def getProductInfo(productId: Long) = {
val baseUrl = ...
val responseEntity =
asyncRestTemplate.getForEntity(baseUrl +
"/productinfo/{productId}",
classOf[ProductInfo], productId)
toRxObservable(responseEntity).map {
(r : ResponseEntity[ProductInfo]) => r.getBody
}
}

@crichardson
ListenableFuture Observable
implicit def toRxObservable[T](future: ListenableFuture[T])
: Observable[T] = {
def create(o: Observer[T]) = {
future.addCallback(new ListenableFutureCallback[T] {
def onSuccess(result: T) {
o.onNext(result)
o.onCompleted()
}
def onFailure(t: Throwable) {
o.onError(t)
}
})
new Subscription {
def unsubscribe() {}
}
}
Observable.create(create _)
}
Supply single
value
Indicate failure
Do nothing

@crichardson
Rx - ProductDetailsService
@Component
class ProductDetailsService @Autowired()
(productInfoService: ProductInfoService,
reviewService: ReviewService,
ecommendationService: RecommendationService) {
def getProductDetails(productId: Long) = {
val productInfo = productInfoService.getProductInfo(productId)
val recommendations =
recommendationService.getRecommendations(productId)
val reviews = reviewService.getReviews(productId)
Observable.zip(productInfo, recommendations, reviews,
(p : ProductInfo, r : Recommendations, rv : Reviews) =>
ProductDetails(p, r, rv))
}
}
Just like Scala Futures

@crichardson
Rx - ProductController
@Controller
class ProductController
@ResponseBody
def productDetails(@PathVariable productId: Long) =
val pd =
toDeferredResult(pd)

@crichardson
Rx - Observable
DeferredResult
implicit def toDeferredResult[T](observable : Observable[T]) = {
val result = new DeferredResult[T]
observable.subscribe(new Observer[T] {
def onCompleted() {}
def onError(e: Throwable) {
result.setErrorResult(e)
}
def onNext(r: T) {
result.setResult(r.asInstanceOf[T])
}
})
result
}

@crichardson
RxObservables vs. Futures
Much better than JDK 7 futures
Comparable to Scala Futures
Rx Scala code is slightly more verbose

@crichardson
Making this code robust
Hystrix works with Observables (and thread pools)
But
For event-driven code we are on our own

@crichardson
Back to the stream of Trades
averaging example...

@crichardson
Calculating averages
Observable<AveragePrice>
calculateAverages(Observable<Trade> trades) {
...
}
class Trade {
private String symbol;
private double price;
class AveragePrice {
private String symbol;
private
double averagePrice;

@crichardson
Using groupBy()
APPL : 401 IBM : 405 CAT : 405 APPL: 403
groupBy( (trade) => trade.symbol)
APPL : 401
IBM : 405
CAT : 405 ...
APPL: 403
Observable<GroupedObservable<String, Trade>>
Observable<Trade>

@crichardson
Using window()
APPL : 401 APPL : 405 APPL : 405 ...
APPL : 401 APPL : 405 APPL : 405
APPL : 405 APPL : 405 APPL : 403
APPL : 405 ...
window(...)
Observable<Trade>
Observable<Observable<Trade>>
N secs
N secs
M secs

@crichardson
Using fold()
APPL : 402 APPL : 405 APPL : 405
APPL : 404
fold(0)(sumCalc) / length
Observable<Trade>
Observable<AveragePrice> Singleton

@crichardson
Using merge()
merge()
APPL : 401
IBM : 405
CAT : 405 ...
APPL: 403
APPL : 401 IBM : 405 CAT : 405 APPL: 403
Observable<Observable<AveragePrice>>
Observable<AveragePrice>

@crichardson
Calculating average prices
def calculateAverages(trades: Observable[Trade]): Observable[AveragePrice] = {
trades.groupBy(_.symbol).map { symbolAndTrades =>
val (symbol, tradesForSymbol) = symbolAndTrades
...
tradesForSymbol.window(...).map {
windowOfTradesForSymbol =>
windowOfTradesForSymbol.fold((0.0, 0, List[Double]())) { (soFar, trade) =>
val (sum, count, prices) = soFar
(sum + trade.price, count + 1, trade.price +: prices)
} map { x =>
val (sum, length, prices) = x
AveragePrice(symbol, sum / length, prices)
}
}.flatMap(identity)
}.flatMap(identity)
}

@crichardson
RxJS / NodeJS example
var rx = require("rx");
function tailFile (fileName) {
var self = this;
var o = rx.Observable.create(function (observer) {
var watcher = self.tail(fileName, function (line) {
observer.onNext(line);
});
return function () {
watcher.close();
}
});
return o.map(parseLogLine);
}
function parseLogLine(line) { ... }

@crichardson
Rx in the browser -
implementing completion
TextChanges(input)
.DistinctUntilChanged()
.Throttle(TimeSpan.FromMilliSeconds(10))
.Select(word=>Completions(word))
.Switch()
.Subscribe(ObserveChanges(output));
Your Mouse is a Database
http://queue.acm.org/detail.cfm?id=2169076
Ajax call returning
Observable
Change events Observable
Display completions

@crichardson
Summary
Consuming web services asynchronously is essential
Scala-style are a powerful concurrency abstraction
Rx Observables are even more powerful
Rx Observables are a unifying abstraction for a wide
variety of use cases

@crichardson
Questions?
@crichardson chris@chrisrichardson.net
http://plainoldobjects.com

Consuming web services asynchronously with Futures and Rx Observables (svcc, svcc2013)

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (20)

Ähnlich wie Consuming web services asynchronously with Futures and Rx Observables (svcc, svcc2013)

Ähnlich wie Consuming web services asynchronously with Futures and Rx Observables (svcc, svcc2013) (20)

Mehr von Chris Richardson

Mehr von Chris Richardson (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Consuming web services asynchronously with Futures and Rx Observables (svcc, svcc2013)