Slides from my JDK.IO 2017 talk "Enabling Googley microservices with gRPC"

1. Alex Borysov
Software Engineer
Enabling Googley microservices with gRPC.
A high performance, open source, HTTP/2-based RPC framework.
JDK.IO, June 19, 2017

2. Google Cloud Platform
Alex Borysov
• Software engineer experienced in large scale software
development.
• 10+ years of software engineering experience.
• Active gRPC user.
@aiborisov

3. Google Cloud Platform
What is gRPC?
gRPC stands for gRPC Remote Procedure Calls.
A high performance, open source, general purpose
standards-based, feature-rich RPC framework.
Open sourced version of Stubby RPC used in Google.
Actively developed and production-ready, current
version is 1.4.0.
#JDKIO @aiborisov

4. Google Cloud Platform
Why gRPC?
Photo taken by Andrey Borisenko.
#JDKIO @aiborisov

5. Google Cloud Platform
gRPC vs JSON/HTTP for Google Cloud Pub/Sub
3x increase in throughput
https://cloud.google.com/blog/big-data/2016/03/announcing-grpc-alpha-for-google-cloud-pubsub
11x difference per CPU
#JDKIO @aiborisov

6. Google Cloud Platform
Google ~O(1010
) QPS.
#JDKIO @aiborisov

7. Google Cloud Platform
Google ~O(1010
) QPS.
Your project QPS?
#JDKIO @aiborisov

8. Google Cloud Platform
Continuous Performance Benchmarking
http://www.grpc.io/docs/guides/benchmarking.html
8 core VMs, unary
#JDKIO @aiborisov
32 core VMs, unary

9. Google Cloud Platform
• Single TCP connection.
• No Head-of-line blocking.
• Binary framing layer.
• Header Compression.
HTTP/2 in One Slide
Transport(TCP)
Application (HTTP/2)
Network (IP)
Session (TLS) [optional]
Binary Framing
HEADERS Frame
DATA Frame
HTTP/2
POST: /upload
HTTP/1.1
Host: www.jdk.io
Content-Type: application/json
Content-Length: 27
HTTP/1.x
{“msg”: “Welcome to 2017!”}
#JDKIO @aiborisov

10. Google Cloud Platform
HTTP/1.x vs HTTP/2
http://www.http2demo.io/
https://http2.golang.org/gophertiles
#JDKIO @aiborisov
HTTP/1.1 HTTP/2

11. Google Cloud Platform
Okay, but how?
#JDKIO @aiborisov

12. Google Cloud Platform
Service Definition (weather.proto)
#JDKIO @aiborisov

13. Google Cloud Platform
Service Definition (weather.proto)
syntax = "proto3";
option java_multiple_files = true;
option java_package = "io.jdk.grpcexample";
package io.jdk.grpcexample;
service WeatherService {
rpc GetCurrent(WeatherRequest) returns (WeatherResponse);
}
#JDKIO @aiborisov

14. Google Cloud Platform
Service Definition (weather.proto)
syntax = "proto3";
option java_multiple_files = true;
option java_package = "io.jdk.grpcexample";
package io.jdk.grpcexample;
service WeatherService {
rpc GetCurrent(WeatherRequest) returns (WeatherResponse);
}
#JDKIO @aiborisov

15. Google Cloud Platform
Service Definition (weather.proto)
syntax = "proto3";
service WeatherService {
rpc GetCurrent(WeatherRequest) returns (WeatherResponse);
}
message WeatherRequest {
Coordinates coordinates = 1;
message Coordinates {
fixed64 latitude = 1;
fixed64 longitude = 2;
}
}
message WeatherResponse {
Temperature temperature = 1;
Humidity humidity = 2;
}
message Temperature {
float degrees = 1;
Units units = 2;
enum Units {
FAHRENHEIT = 0;
CELSIUS = 1;
}
}
message Humidity {
float value = 1;
}
#JDKIO @aiborisov

16. Google Cloud Platform
Generated Classes
$ ls -1 src/generated/main/java/io/jdk/grpcdemo/
Coordinates.java
CoordinatesOrBuilder.java
Humidity.java
HumidityOrBuilder.java
Temperature.java
TemperatureOrBuilder.java
WeatherRequest.java
WeatherRequestOrBuilder.java
WeatherResponse.java
WeatherResponseOrBuilder.java
#JDKIO @aiborisov

17. Google Cloud Platform
Generated Classes
$ ls -1 src/generated/main/java/io/jdk/grpcdemo/
Coordinates.java
CoordinatesOrBuilder.java
Humidity.java
HumidityOrBuilder.java
Temperature.java
TemperatureOrBuilder.java
WeatherRequest.java
WeatherRequestOrBuilder.java
WeatherResponse.java
WeatherResponseOrBuilder.java
$ ls -1 src/generated/main/grpc/io/jdk/grpcdemo/
WeatherServiceGrpc.java
#JDKIO @aiborisov

18. Google Cloud Platform
Implement gRPC Service
public class WeatherService extends WeatherServiceGrpc.WeatherServiceImplBase {
@Override
public void getCurrent(WeatherRequest request,
StreamObserver<WeatherResponse> responseObserver) {
}
}
Abstract class
#JDKIO @aiborisov

19. Google Cloud Platform
Implement gRPC Service
public class WeatherService extends WeatherServiceGrpc.WeatherServiceImplBase {
@Override
public void getCurrent(WeatherRequest request,
public interface StreamObserver<WeatherResponse> responseObserver){{
void onNext(WeatherResponse response);
void onCompleted();
void onError(Throwable error);
}
}
}
#JDKIO @aiborisov

20. Google Cloud Platform
Implement gRPC Service
public class WeatherService extends WeatherServiceGrpc.WeatherServiceImplBase {
@Override
public void getCurrent(WeatherRequest request,
StreamObserver<WeatherResponse> responseObserver) {
WeatherResponse response = WeatherResponse.newBuilder()
.setTemperature(Temperature.newBuilder().setUnits(CELSUIS).setDegrees(20.f))
.setHumidity(Humidity.newBuilder().setValue(.65f))
.build();
responseObserver.onNext(response);
responseObserver.onCompleted();
}
}
#JDKIO @aiborisov

21. Google Cloud Platform
Start gRPC Server
void start() throws IOException {
Server grpcServer = NettyServerBuilder.forPort(8090)
.addService(new WeatherService()).build()
.start();
Runtime.getRuntime().addShutdownHook(new Thread(grpcServer::shutdown));
grpcServer.awaitTerminaton();
}
#JDKIO @aiborisov

22. Google Cloud Platform
gRPC Clients
ManagedChannel grpcChannel = NettyChannelBuilder.forAddress("localhost", 8090).build();
WeatherServiceStub client = WeatherServiceGrpc.newStub(grpcChannel);
#JDKIO @aiborisov

23. Google Cloud Platform
gRPC Clients
ManagedChannel grpcChannel = NettyChannelBuilder.forAddress("localhost", 8090).build();
WeatherServiceStub client = WeatherServiceGrpc.newStub(grpcChannel);
WeatherServiceBlockingStub blockingClient = WeatherServiceGrpc.newBlockingStub(grpcChannel);
WeatherServiceFutureStub futureClient = WeatherServiceGrpc.newFutureStub(grpcChannel);
#JDKIO @aiborisov

24. Google Cloud Platform
gRPC Sync Client
WeatherRequest request = WeatherRequest.newBuilder()
.setCoordinates(Coordinates.newBuilder().setLatitude(420000000)
.setLongitude(-720000000)).build();
WeatherResponse response = blockingClient.getCurrent(request);
logger.info("Current weather for {}: {}", request, response);
#JDKIO @aiborisov

25. Google Cloud Platform
gRPC Async Client
WeatherRequest request = WeatherRequest.newBuilder()
.setCoordinates(Coordinates.newBuilder().setLatitude(504316220)
.setLongitude(305166450)).build();
client.getCurrent(request, new StreamObserver<WeatherResponse>() {
@Override
public void onNext(WeatherResponse response) {
logger.info("Current weather for {}: {}", request, response);
}
@Override
public void onError(Throwable t) { logger.info("Cannot get weather for {}", request); }
@Override
public void onCompleted() { logger.info("Stream completed."); }
});
#JDKIO @aiborisov

26. Google Cloud Platform
Implement gRPC Service
public class WeatherService extends WeatherServiceGrpc.WeatherServiceImplBase {
@Override
public void getCurrent(WeatherRequest request,
StreamObserver<WeatherResponse> responseObserver) {
WeatherResponse response = WeatherResponse.newBuilder()
.setTemperature(Temperature.newBuilder().setUnits(CELSUIS).setDegrees(20.f))
.setHumidity(Humidity.newBuilder().setValue(.65f))
.build();
responseObserver.onNext(response);
responseObserver.onCompleted();
}
}
#JDKIO @aiborisov

27. Google Cloud Platform
Adding New [Micro]Services
Weather
Weather
Temp
Wind
Humidity
#JDKIO @aiborisov

28. Google Cloud Platform
Service Definitions
service WeatherService {
rpc GetCurrent(WeatherRequest) returns (WeatherResponse);
}
service TemperatureService {
rpc GetCurrent(Coordinates) returns (Temperature);
}
service HumidityService {
rpc GetCurrent(Coordinates) returns (Humidity);
}
service WindService {
rpc GetCurrent(Coordinates) returns (Wind);
}
#JDKIO @aiborisov

29. Google Cloud Platform
Service Definition - Response Message
message WeatherResponse {
Temperature temperature = 1;
float humidity = 2;
}
#JDKIO @aiborisov

30. Google Cloud Platform
Adding New Field
message WeatherResponse {
Temperature temperature = 1;
float humidity = 2;
Wind wind = 3;
}
message Wind {
Speed speed = 1;
float direction = 2;
}
message Speed {
float value = 1;
Units units = 2;
enum Units {
MPH = 0;
MPS = 1;
KNOTS = 2;
KMH = 3;
}
}
#JDKIO @aiborisov

31. Google Cloud Platform
private final TemperatureServiceBlockingStub temperatureService;
private final HumidityServiceBlockingStub humidityService;
private final WindServiceBlockingStub windService;
public WeatherService(TemperatureServiceBlockingStub temperatureService,
HumidityServiceBlockingStub humidityService,
WindServiceBlockingStub windService) {
this.temperatureService = temperatureService;
this.humidityService = humidityService;
this.windService = windService;
}
...
Blocking Stub Dependencies
public class WeatherService extends WeatherServiceGrpc.WeatherServiceImplBase {
#JDKIO @aiborisov

32. Google Cloud Platform
...
@Override
public void getCurrent(WeatherRequest request,
StreamObserver<WeatherResponse> responseObserver) {
Temperature temperature = temperatureService.getCurrent(request.getCoordinates());
Humidity humidity = humidityService.getCurrent(request.getCoordinates());
Wind wind = windService.getCurrent(request.getCoordinates());
WeatherResponse response = WeatherResponse.newBuilder()
.setTemperature(temperature).setHumidity(humidity).setWind(wind).build();
responseObserver.onNext(response);
responseObserver.onCompleted();
}
}
Blocking Stub Dependencies
public class WeatherService extends WeatherServiceGrpc.WeatherServiceImplBase {
#JDKIO @aiborisov

33. 33
Blocking Stubs
Temp
Humidity
Wind
#JDKIO @aiborisov

34. Google Cloud Platform
private final TemperatureServiceFutureStub tempService;
private final HumidityServiceFutureStub humidityService;
private final WindServiceFutureStub windService;
public WeatherService(TemperatureServiceFutureStub tempService,
HumidityServiceFutureStub humidityService,
WindServiceFutureStub windService) {
this.tempService = tempService;
this.humidityService = humidityService;
this.windService = windService;
}
...
Async Future Stub Dependencies
public class WeatherService extends WeatherServiceGrpc.WeatherServiceImplBase {
#JDKIO @aiborisov

35. Google Cloud Platform
@Override
public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) {
Coordinates coordinates = request.getCoordinates();
ListenableFuture<List<WeatherResponse>> responsesFuture = Futures.allAsList(
Futures.transform(tempService.getCurrent(coordinates),
(Temperature temp) -> WeatherResponse.newBuilder().setTemperature(temp).build()),
Futures.transform(windService.getCurrent(coordinates),
(Wind wind) -> WeatherResponse.newBuilder().setWind(wind).build()),
Futures.transform(humidityService.getCurrent(coordinates),
(Humidity humidity) -> WeatherResponse.newBuilder().setHumidity(humidity).build())
);
...
Async Future Stub Dependencies
public class WeatherService extends WeatherServiceGrpc.WeatherServiceImplBase {
#JDKIO @aiborisov

36. Google Cloud Platform
@Override
public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) {
Coordinates coordinates = request.getCoordinates();
ListenableFuture<List<WeatherResponse>> responsesFuture = Futures.allAsList(
Futures.transform(tempService.getCurrent(coordinates),
(Temperature temp) -> WeatherResponse.newBuilder().setTemperature(temp).build()),
Futures.transform(windService.getCurrent(coordinates),
(Wind wind) -> WeatherResponse.newBuilder().setWind(wind).build()),
Futures.transform(humidityService.getCurrent(coordinates),
(Humidity humidity) -> WeatherResponse.newBuilder().setHumidity(humidity).build())
);
...
Async Future Stub Dependencies
public class WeatherService extends WeatherServiceGrpc.WeatherServiceImplBase {
#JDKIO @aiborisov

37. Google Cloud Platform
@Override
public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) {
Coordinates coordinates = request.getCoordinates();
ListenableFuture<List<WeatherResponse>> responsesFuture = Futures.allAsList(
Futures.transform(tempService.getCurrent(coordinates),
(Temperature temp) -> WeatherResponse.newBuilder().setTemperature(temp).build()),
Futures.transform(windService.getCurrent(coordinates),
(Wind wind) -> WeatherResponse.newBuilder().setWind(wind).build()),
Futures.transform(humidityService.getCurrent(coordinates),
(Humidity humidity) -> WeatherResponse.newBuilder().setHumidity(humidity).build())
);
...
Async Future Stub Dependencies
public class WeatherService extends WeatherServiceGrpc.WeatherServiceImplBase {
#JDKIO @aiborisov

38. Google Cloud Platform
...
Futures.addCallback(responsesFuture, new FutureCallback<List<WeatherResponse>>() {
@Override
public void onSuccess(@Nullable List<WeatherResponse> results) {
WeatherResponse.Builder response = WeatherResponse.newBuilder();
results.forEach(response::mergeFrom);
responseObserver.onNext(response.build());
responseObserver.onCompleted();
}
@Override
public void onFailure(Throwable t) { responseObserver.onError(t); }
});
}
}
Async Future Stub Dependencies
public class WeatherService extends WeatherServiceGrpc.WeatherServiceImplBase {
@Override
public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) {
#JDKIO @aiborisov

39. Google Cloud Platform
Netty Transport
ManagedChannel grpcChannel = NettyChannelBuilder.forAddress("localhost", 8090).build();
WeatherServiceFutureStub futureClient = WeatherServiceGrpc.newFutureStub(grpcChannel);
#JDKIO @aiborisov

40. Google Cloud Platform
Netty Transport
ManagedChannel grpcChannel = NettyChannelBuilder.forAddress("localhost", 8090).build();
WeatherServiceFutureStub futureClient = WeatherServiceGrpc.newFutureStub(grpcChannel);
Server grpcServer = NettyServerBuilder.forPort(8090)
.addService(new WeatherService()).build().start();
#JDKIO @aiborisov

41. Google Cloud Platform
Netty: Asynchronous and Non-Blocking IO
Netty-based transport:
• Multiplexes conn-s on the event loops: EpollEventLoopGroup, NioEventLoopGroup.
• Decouples I/O waiting from threads.
• gRPC uses Netty event loops for both client and server transports.
Request
Event
Loop
Worker thread
Worker thread
Worker thread
Worker thread
Worker thread
Worker thread
Worker thread
Event
Loop
Event
Loop
Event
Loop
Callback
Request
Callback
N
e
t
w
o
r
k
N
e
t
w
o
r
k
#JDKIO @aiborisov

42. Google Cloud Platform
Async Future Stub Dependencies
public class WeatherService extends WeatherServiceGrpc.WeatherServiceImplBase {
@Override
public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) {
...
Futures.addCallback(responsesFuture, new FutureCallback<List<WeatherResponse>>() {
@Override
public void onSuccess(@Nullable List<WeatherResponse> results) {
WeatherResponse.Builder response = WeatherResponse.newBuilder();
results.forEach(response::mergeFrom);
responseObserver.onNext(response.build());
responseObserver.onCompleted();
}
@Override
public void onFailure(Throwable t) { responseObserver.onError(t); }
});
}
}
#JDKIO @aiborisov

43. Google Cloud Platform
Async Future Stub Dependencies
public class WeatherService extends WeatherServiceGrpc.WeatherServiceImplBase {
@Override
public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) {
...
Futures.addCallback(responsesFuture, new FutureCallback<List<WeatherResponse>>() {
@Override
public void onSuccess(@Nullable List<WeatherResponse> results) {
WeatherResponse.Builder response = WeatherResponse.newBuilder();
results.forEach(response::mergeFrom);
responseObserver.onNext(response.build());
responseObserver.onCompleted();
}
@Override
public void onFailure(Throwable t) { responseObserver.onError(t); }
});
}
}
#JDKIO @aiborisov

44. Google Cloud Platform
Panta rhei, "everything flows".
Heraclitus of Ephesus
~2,400 years Before HTTP/1.x
https://en.wikipedia.org/wiki/Heraclitus
“ ”
#JDKIO @aiborisov

45. Google Cloud Platform
Service Definitions
service WeatherService {
rpc GetCurrent(WeatherRequest) returns (WeatherResponse);
}
service TemperatureService {
rpc GetCurrent(Coordinates) returns (Temperature);
}
service HumidityService {
rpc GetCurrent(Coordinates) returns (Humidity);
}
service WindService {
rpc GetCurrent(Coordinates) returns (Wind);
}
#JDKIO @aiborisov

46. Google Cloud Platform
Streaming Service Definitions
service WeatherStreamingService {
rpc GetCurrent(WeatherRequest) returns (stream WeatherResponse);
}
service TemperatureStreamingService {
rpc GetCurrent(Coordinates) returns (stream Temperature);
}
service HumidityStreamingService {
rpc GetCurrent(Coordinates) returns (stream Humidity);
}
service WindStreamingService {
rpc GetCurrent(Coordinates) returns (stream Wind);
}
#JDKIO @aiborisov

47. Google Cloud Platform
Bidirectional Streaming Service Definitions
service WeatherStreamingService {
rpc GetCurrent(stream WeatherRequest) returns (stream WeatherResponse);
}
service TemperatureStreamingService {
rpc GetCurrent(stream Coordinates) returns (stream Temperature);
}
service HumidityStreamingService {
rpc GetCurrent(stream Coordinates) returns (stream Humidity);
}
service WindStreamingService {
rpc GetCurrent(stream Coordinates) returns (stream Wind);
}
#JDKIO @aiborisov

48. Google Cloud Platform
Bidirectional Streaming Service Definitions
service WeatherStreamingService {
rpc Observe(stream WeatherRequest) returns (stream WeatherResponse);
}
service TemperatureStreamingService {
rpc Observe(stream Coordinates) returns (stream Temperature);
}
service HumidityStreamingService {
rpc Observe(stream Coordinates) returns (stream Humidity);
}
service WindStreamingService {
rpc Observe(stream Coordinates) returns (stream Wind);
}
#JDKIO @aiborisov

49. Google Cloud Platform
Bidirectional Streaming Service
public class WeatherStreamingService extends WeatherStreamingGrpc.WeatherStreamingImplBase {
...
@Override
public StreamObserver<WeatherRequest> observe(StreamObserver<WeatherResponse> responseObserver) {
#JDKIO @aiborisov

50. Google Cloud Platform
Bidirectional Streaming Service
public class WeatherStreamingService extends WeatherStreamingGrpc.WeatherStreamingImplBase {
...
@Override
public StreamObserver<WeatherRequest> observe(StreamObserver<WeatherResponse> responseObserver) {
StreamObserver<Coordinates> temperatureClientStream =
temperatureService.observe(new StreamObserver<Temperature>() {
@Override
public void onNext(Temperature temperature) {
WeatherResponse resp = WeatherResponse.newBuilder().setTemperature(temperature).build()
responseObserver.onNext(resp);
}
@Override
public void onError(Throwable t) { responseObserver.onError(t); }
@Override
public void onCompleted() { responseObserver.onCompleted(); } });
...
}
#JDKIO @aiborisov

51. Google Cloud Platform
Bidirectional Streaming Service
public class WeatherStreamingService extends WeatherStreamingGrpc.WeatherStreamingImplBase {
...
@Override
public StreamObserver<WeatherRequest> observe(StreamObserver<WeatherResponse> responseObserver) {
StreamObserver<Coordinates> temperatureClientStream =
temperatureService.observe(new StreamObserver<Temperature>() {
@Override
public void onNext(Temperature temperature) {
WeatherResponse resp = WeatherResponse.newBuilder().setTemperature(temperature).build()
responseObserver.onNext(resp);
}
@Override
public void onError(Throwable t) { responseObserver.onError(t); }
@Override
public void onCompleted() { responseObserver.onCompleted(); } });
...
}
#JDKIO @aiborisov

52. Google Cloud Platform
return new StreamObserver<WeatherRequest>() {
@Override
public void onNext(WeatherRequest request) {
temperatureClientStream.onNext(request.getCoordinates());
}
@Override
public void onError(Throwable t) { temperatureClientStream.onError(t); }
@Override
public void onCompleted() { temperatureClientStream.onCompleted(); }
};
}
Bidirectional Streaming Service
public class WeatherStreamingService extends WeatherStreamingGrpc.WeatherStreamingImplBase {
...
@Override
public StreamObserver<WeatherRequest> observe(StreamObserver<WeatherResponse> responseObserver) {
StreamObserver<Coordinates> temperatureClientStream = ...
#JDKIO @aiborisov

53. 53
Messaging applications.
Games / multiplayer tournaments.
Moving objects.
Sport results.
Stock market quotes.
Smart home devices.
You name it!
Streaming Use-Cases
#JDKIO @aiborisov

54. Google Cloud Platform
Continuous Performance Benchmarking
http://www.grpc.io/docs/guides/benchmarking.html
8 core VMs, streaming
#JDKIO @aiborisov
32 core VMs, streaming

55. Google Cloud Platform
gRPC Speaks Your Language
● Java
● Go
● C/C++
● C#
● Node.js
● PHP
● Ruby
● Python
● Objective-C
● MacOS
● Linux
● Windows
● Android
● iOS
Service definitions and client libraries Platforms supported
#JDKIO @aiborisov

56. Google Cloud Platform
Interoperability
Java
Service
Python
Service
GoLang
Service
C++
Service
gRPC
Service
gRPC
Stub
gRPC
Stub
gRPC
Stub
gRPC
Stub
gRPC
Service
gRPC
Service
gRPC
Service
gRPC
Stub
#JDKIO @aiborisov

57. #JDKIO @aiborisov
Fault Tolerance?

58. Google Cloud Platform
Use time-outs!
And it will be fine.
“ ”
#JDKIO @aiborisov

59. Google Cloud Platform
Timeouts?
GW
A1 A2 A3
B1 B2 B3
C1 C2 C3
200 ms
?
?
?
? ?
? ?
? ?
#JDKIO @aiborisov

60. Google Cloud Platform
Default Timeout For Every Service?
GW
A1 A2 A3
B1 B2 B3
C1 C2 C3
200 ms
200 ms
200 ms
200 ms
200 ms 200 ms
200 ms 200 ms
200 ms 200 ms
#JDKIO @aiborisov

61. Google Cloud Platform
Default Timeout For Every Service?
Gateway
20 ms 10 ms 10 ms
FastFastFast
30 ms 40 ms 20 ms
130 ms
200 ms
timeout
200 ms
timeout
200 ms
timeout
200 ms
timeout
#JDKIO @aiborisov

62. Google Cloud Platform
Default Timeout For Every Service?
Gateway
30 ms 80 ms 100 ms
SlowFair
210 ms
200 ms
timeout
200 ms
timeout
200 ms
timeout
200 ms
timeout
#JDKIO @aiborisov

63. Google Cloud Platform
Default Timeout For Every Service?
30 ms 80 ms 100 ms
SlowFair
210 ms
200 ms
timeout
200 ms
timeout
200 ms
timeout
200 ms
timeout
#JDKIO @aiborisov

64. Google Cloud Platform
Default Timeout For Every Service?
80 ms 100 ms
SlowFair
200 ms
timeout
200 ms
timeout
200 ms
timeout
200 ms
timeout
Still working!
#JDKIO @aiborisov

65. Google Cloud Platform
Default Timeout For Every Service?
BusyBusy
200 ms
timeout
200 ms
timeout
200 ms
timeout
Idle Busy
#JDKIO @aiborisov

66. Google Cloud Platform
Default Timeout For Every Service?
BusyBusy
200 ms
timeout
200 ms
timeout
200 ms
timeout
Idle Busy
R
e
t
r
y
#JDKIO @aiborisov

67. Google Cloud Platform
Default Timeout For Every Service?
BusyBusy
200 ms
timeout
200 ms
timeout
200 ms
timeout
Idle Busy
R
e
t
r
y
#JDKIO @aiborisov

68. Google Cloud Platform
Default Timeout For Every Service?
200 ms
timeout
200 ms
timeout
200 ms
timeout
R
e
t
r
y
#JDKIO @aiborisov

69. Google Cloud Platform
Tune Timeout For Every Service?
GW
A1 A2 A3
B1 B2 B3
C1 C2 C3
200 ms
190 ms
190 ms
190 ms
110 ms 50 ms
110 ms 50 ms
110 ms 50 ms
#JDKIO @aiborisov

70. Google Cloud Platform
Tune Timeout For Every Service?
Gateway
200 ms
timeout
190 ms
timeout
110 ms
timeout
30 ms 80 ms 25 ms
FastFair
55 ms
Fast
30 ms
50 ms
timeout
#JDKIO @aiborisov

71. Google Cloud Platform
Tune Timeout For Every Service?
Gateway
200 ms
timeout
190 ms
timeout
110 ms
timeout
30 ms 80 ms 25 ms
Fair
55 ms
Fast
30 ms
50 ms
timeout
#JDKIO @aiborisov

72. Google Cloud Platform
Timeouts?
GW
A1 A2 A3
B1 B2 B3
C1 C2 C3
200 ms
?
?
?
? ?
? ?
? ?
#JDKIO @aiborisov

73. Google Cloud Platform
Timeouts for Real World?
GW
A1 A2 A3
B1 B2 B3
C1 C2 C3
200 ms
#JDKIO @aiborisov

74. Google Cloud Platform
Timeouts for Real World?
GW
A1 A2 A3
B1 B2 B3
C1 C2 C3
200 ms
2,500 ms
140 ms
#JDKIO @aiborisov

75. 75
Timeouts in gRPC
#JDKIO @aiborisov

76. 76
gRPC Java does not support timeouts.
Timeouts in gRPC
#JDKIO @aiborisov

77. 77
gRPC Java does not support timeouts.
gRPC supports deadlines instead!
gRPC Deadlines
WeatherResponse response =
client.withDeadlineAfter(200, MILLISECONDS)
.getCurrent(request);
#JDKIO @aiborisov

78. 78
First-class feature in gRPC.
Deadline is an absolute point in time.
Deadline indicates to the server how
long the client is willing to wait for an
answer.
RPC will fail with DEADLINE_EXCEEDED
status code when deadline reached.
gRPC Deadlines
#JDKIO @aiborisov

79. Google Cloud Platform
gRPC Deadline Propagation
Gateway
Now =
1476600000000
Deadline =
1476600000200
Remaining = 200
withDeadlineAfter(200, MILLISECONDS)
DEADLINE_EXCEEDED DEADLINE_EXCEEDED
60 ms
DEADLINE_EXCEEDED
90 ms
DEADLINE_EXCEEDED
60 ms
Now =
1476600000060
Deadline =
1476600000200
Remaining = 140
Now =
1476600000150
Deadline =
1476600000200
Remaining = 50
Now =
1476600000210
Deadline =
1476600000200
Remaining = -10
#JDKIO @aiborisov

80. 80
Deadlines are automatically propagated!
Can be accessed by the receiver!
gRPC Deadlines
Context context = Context.current();
context.getDeadline().isExpired();
context.getDeadline()
.timeRemaining(MILLISECONDS);
context.getDeadline().runOnExpiration(() ->
logger.info("Deadline exceeded!"), exec);
#JDKIO @aiborisov

81. 81
Deadlines are expected.
What about unpredictable cancellations?
• User cancelled request.
• Caller is not interested in the result any
more.
• etc
Cancellation?
#JDKIO @aiborisov

82. Google Cloud Platform
Cancellation?
GW
Busy Busy Busy
Busy Busy Busy
Busy Busy Busy
RPC
Active RPC Active RPC
Active RPC
Active RPC Active RPCActive RPC
Active RPC Active RPC
Active RPC
#JDKIO @aiborisov

83. Google Cloud Platform
Cancellation?
GW
Busy Busy Busy
Busy Busy Busy
Busy Busy Busy
Active RPC Active RPC
Active RPC
Active RPC Active RPCActive RPC
Active RPC Active RPC
Active RPC
#JDKIO @aiborisov

84. Google Cloud Platform
Cancellation Propagation
GW
Idle Idle Idle
Idle Idle Idle
Idle Idle Idle
#JDKIO @aiborisov

85. 85
Automatically propagated.
RPC fails with CANCELLED status code.
Cancellation status can be accessed by
the receiver.
Server (receiver) always knows if RPC is
valid!
gRPC Cancellation
#JDKIO @aiborisov

86. Google Cloud Platform
gRPC Cancellation
public class WeatherService extends WeatherGrpc.WeatherServiceImplBase {
...
@Override
public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) {
ServerCallStreamObserver<WeatherResponse> streamObserver =
(ServerCallStreamObserver<WeatherResponse>) responseObserver;
streamObserver.isCancelled();
...
#JDKIO @aiborisov

87. Google Cloud Platform
gRPC Cancellation
public class WeatherService extends WeatherGrpc.WeatherServiceImplBase {
...
@Override
public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) {
ServerCallStreamObserver<WeatherResponse> streamObserver =
(ServerCallStreamObserver<WeatherResponse>) responseObserver;
streamObserver.setOnCancelHandler(() -> {
cleanupResources();
logger.info("Call cancelled by client!");
});
...
#JDKIO @aiborisov

88. Google Cloud Platform
gRPC Context
public class WeatherService extends WeatherGrpc.WeatherServiceImplBase {
...
@Override
public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) {
Context.current().getDeadline();
Context.current().isCancelled();
Context.current().cancellationCause();
Context.current().addListener(context -> {
cleanupResources();
logger.info("Call cancelled by client!"); }, executor)
...
#JDKIO @aiborisov

89. Google Cloud Platform
More Control?
#JDKIO @aiborisov

90. Google Cloud Platform
BiDi Streaming - Slow Client
Fast Server
Request
Responses
Slow Client
CANCELLED
UNAVAILABLE
RESOURCE_EXHAUSTED
#JDKIO @aiborisov

91. Google Cloud Platform
BiDi Streaming - Slow Server
Slow Server
Request
Response
Fast Client
CANCELLED
UNAVAILABLE
RESOURCE_EXHAUSTED
Requests
#JDKIO @aiborisov

92. Google Cloud Platform
Client-Side Flow-Control
Server
Request, count = 4
Client
2 Responses
4 Responses
Count = 2
#JDKIO @aiborisov

93. Google Cloud Platform
Server-Side Flow-Control
Server
Request
Client
Response
count = 2
2 Requests
count = 3
3 Requests
#JDKIO @aiborisov

94. Google Cloud Platform
Flow-Control (Client-Side)
CallStreamObserver<WeatherRequest> requestStream =
(CallStreamObserver) client.observe(new ClientResponseObserver<WeatherRequest, WeatherResponse>() {
@Override
public void beforeStart(ClientCallStreamObserver outboundStream) {
outboundStream.disableAutoInboundFlowControl();
}
@Override
public void onNext(WeatherResponse response) { processResponse(response); }
@Override
public void onError(Throwable e) { logger.error("Error on weather request.", e); }
@Override
public void onCompleted() { logger.info("Stream completed."); }
});
requestStream.onNext(request);
requestStream.request(3); // Request up to 3 responses from server
#JDKIO @aiborisov

95. Google Cloud Platform
Flow-Control (Client-Side)
CallStreamObserver<WeatherRequest> requestStream =
(CallStreamObserver) client.observe(new ClientResponseObserver<WeatherRequest, WeatherResponse>() {
@Override
public void beforeStart(ClientCallStreamObserver outboundStream) {
outboundStream.disableAutoInboundFlowControl();
}
@Override
public void onNext(WeatherResponse response) { processResponse(response); }
@Override
public void onError(Throwable e) { logger.error("Error on weather request.", e); }
@Override
public void onCompleted() { logger.info("Stream completed."); }
});
requestStream.onNext(request);
requestStream.request(3); // Request up to 3 responses from server
#JDKIO @aiborisov

96. Google Cloud Platform
Flow-Control (Client-Side)
CallStreamObserver<WeatherRequest> requestStream =
(CallStreamObserver) client.observe(new ClientResponseObserver<WeatherRequest, WeatherResponse>() {
@Override
public void beforeStart(ClientCallStreamObserver outboundStream) {
outboundStream.disableAutoInboundFlowControl();
}
@Override
public void onNext(WeatherResponse response) { processResponse(response); }
@Override
public void onError(Throwable e) { logger.error("Error on weather request.", e); }
@Override
public void onCompleted() { logger.info("Stream completed."); }
});
requestStream.onNext(request);
requestStream.request(3); // Request up to 3 responses from server
#JDKIO @aiborisov

97. Google Cloud Platform
Flow-Control (Client-Side)
CallStreamObserver<WeatherRequest> requestStream =
(CallStreamObserver) client.observe(new ClientResponseObserver<WeatherRequest, WeatherResponse>() {
@Override
public void beforeStart(ClientCallStreamObserver outboundStream) {
outboundStream.disableAutoInboundFlowControl();
}
@Override
public void onNext(WeatherResponse response) { processResponse(response); }
@Override
public void onError(Throwable e) { logger.error("Error on weather request.", e); }
@Override
public void onCompleted() { logger.info("Stream completed."); }
});
requestStream.onNext(request);
requestStream.request(3); // Request up to 3 responses from server
#JDKIO @aiborisov

98. Google Cloud Platform
Flow-Control (Client-Side)
CallStreamObserver<WeatherRequest> requestStream =
(CallStreamObserver) client.observe(new ClientResponseObserver<WeatherRequest, WeatherResponse>() {
@Override
public void beforeStart(ClientCallStreamObserver outboundStream) {
outboundStream.disableAutoInboundFlowControl();
}
@Override
public void onNext(WeatherResponse response) { processResponse(response); }
@Override
public void onError(Throwable e) { logger.error("Error on weather request.", e); }
@Override
public void onCompleted() { logger.info("Stream completed."); }
});
requestStream.onNext(request);
requestStream.request(3); // Request up to 3 responses from server
#JDKIO @aiborisov

99. Google Cloud Platform
Flow-Control (Client-Side)
public class WeatherStreamingService extends WeatherStreamingGrpc.WeatherStreaminServicegImplBase {
...
@Override
public StreamObserver<WeatherRequest> observe(StreamObserver<WeatherResponse> responseObserver) {
ServerCallStreamObserver<WeatherResponse> streamObserver =
(ServerCallStreamObserver<WeatherResponse>) responseObserver;
streamObserver.setOnReadyHandler(() -> {
if (streamObserver.isReady()) {
streamObserver.onNext(calculateWeather());
}
});
...
#JDKIO @aiborisov

100. 100
Helps to balance computing power
and network capacity between client
and server.
gRPC supports both client- and
server-side flow control.
Disabled by default.
Flow-Control
#JDKIO @aiborisov

101. Google Cloud Platform
What else?
#JDKIO @aiborisov

102. 102
Service Discovery
Load Balancing
Tracing, debugging
Monitoring
Testing
Microservices?
#JDKIO @aiborisov

103. Google Cloud Platform
Service Discovery & Load Balancing
HTTP/2
RPC Server-side App
Service Definition
(extends generated definition)
ServerCall handler
TransportTransport
ServerCall
RPC Client-Side App
Channel
Stub
Future
Stub
Blocking
Stub
ClientCall
#JDKIO @aiborisov

104. Google Cloud Platform
HTTP/2
Service Discovery & Load Balancing
RPC Client-Side App
Instance
#1
Instance
#N
Instance
#2
RPC Server-side Apps
Transport
Channel
Stub
Future
Stub
Blocking
Stub
ClientCall
#JDKIO @aiborisov

105. Google Cloud Platform
Service Discovery & Load Balancing
HTTP/2
RPC Client-Side App
Channel
Stub
Future
Stub
Blocking
Stub
ClientCall
RPC Server-side Apps
Tran #1 Tran #2 Tran #N
Service Definition
(extends generated definition)
ServerCall handler
Transport
ServerCall
NameResolver LoadBalancer
Pluggable
Load
Balancing
and
Service
Discovery
#JDKIO @aiborisov

106. Google Cloud Platform
Service Discovery & Load Balancing
ManagedChannel grpcChannel = NettyChannelBuilder.forTarget("WeatherSrv")
.nameResolverFactory(new DnsNameResolverProvider())
.loadBalancerFactory(RoundRobinLoadBalancerFactory.getInstance())
.build();
#JDKIO @aiborisov

107. Google Cloud Platform
Service Discovery & Load Balancing
ManagedChannel grpcChannel = NettyChannelBuilder.forTarget("WeatherSrv")
.nameResolverFactory(new DnsNameResolverProvider())
.loadBalancerFactory(RoundRobinLoadBalancerFactory.getInstance())
.build();
Design documents:
• Load Balancing in gRPC:
https://github.com/grpc/grpc/blob/master/doc/load-balancing.md
• gRPC Java Name Resolution and Load Balancing v2: http://tiny.cc/grpc-java-lb-v2
#JDKIO @aiborisov

108. Google Cloud Platform
Testing Support
In-process server transport and client-side channel
StreamRecorder - StreamObserver that records all the observed values and errors.
MetadataUtils for testing metadata headers and trailer.
grpc-testing - utility functions for unit and integration testing:
https://github.com/grpc/grpc-java/tree/master/testing
#JDKIO @aiborisov

109. Google Cloud Platform
Testing Support - InProcess
In-process transport: fully-featured, high performance, and useful in testing.
WeatherServiceAsync weatherService =
new WeatherServiceAsync(tempService, humidityService, windService);
Server grpcServer = InProcessServerBuilder.forName("weather")
.addService(weatherService).build();
Channel grpcChannel = InProcessChannelBuilder.forName("weather").build();
WeatherServiceBlockingStub stub =
WeatherServiceGrpc.newBlockingStub(grpcChannel).withDeadlineAfter(100, MILLISECONDS);
#JDKIO @aiborisov

110. Google Cloud Platform
More Features
Distributed tracing support:
• OpenTracing, Zipkin / Brave support
Interceptors to add cross-cutting behavior:
• Client- and server-side
Monitoring:
• gRPC Prometheus; grpcz-monitoring; more Monitoring APIs are coming.
Built-in authentication mechanisms:
• SSL/TLS; token-based authentication with Google; authentication API.
Payload compression: https://github.com/grpc/grpc/blob/master/doc/compression.md
#JDKIO @aiborisov

111. Google Cloud Platform
Growing Community and Ecosystem
Your
company could
be here
#JDKIO @aiborisov

112. Google Cloud Platform
Growing Community and Ecosystem
https://github.com/grpc-ecosystem
Polyglot is a universal grpc command line client.
grpc-gateway generates a reverse-proxy server which translates a RESTful JSON API into gRPC.
OpenTracing is a set of consistent, expressive, vendor-neutral APIs for distributed tracing and
context propagation.
Prometheus monitoring support for grpc-java and grpc-go.
#JDKIO @aiborisov

113. Google Cloud Platform
Try It Out!
http://grpc.io
gRPC on Github: https://github.com/grpc
Demo: https://github.com/alxbnet/jdkio-grpc-microservices-demo
Google group: grpc-io@googlegroups.com
gRPC Java quickstart: http://www.grpc.io/docs/quickstart/java.html
gRPC Java tutorial: http://www.grpc.io/docs/tutorials/basic/java.html
gRPC contribution: https://github.com/grpc/grpc-contrib
#JDKIO @aiborisov

114. Google Cloud Platform
Takeaways
Microservices are not free!
gRPC (http://grpc.io):
• HTTP/2 transport based, open source, general purpose
standards-based, feature-rich RPC framework.
• Bidirectional streaming over one single TCP connection.
• Netty transport provides asynchronous and non-blocking I/O.
• Deadline and cancellations propagation.
• Client and server-side flow-control.
• Pluggable service discovery and load-balancing
• Generated client libraries
• Supports 10 programming languages.
• Build-in testing support.
• Production-ready (current version is 1.4.0) and growing ecosystem.
#JDKIO @aiborisov

115. Thank You
#JDKIO @aiborisov

116. Google Cloud Platform
Images Used
Special thanks for the provided photos:
• Andrey Borisenko
• Stanislav Vedmid
Photo from https://www.google.com/about/datacenters page:
• https://www.google.com/about/datacenters/gallery/#/tech/12
Photos licenced by Creative Commons 2.0 https://creativecommons.org/licenses/by/2.0/ :
• https://www.flickr.com/photos/garryknight/5754661212/ by Garry Knight
• https://www.flickr.com/photos/sodaigomi/21128888345/ by sodai gomi
• https://www.flickr.com/photos/zengame/15972170944/ by Zengame
• https://www.flickr.com/photos/londonmatt/18496249193/ by Matt Brown
• https://www.flickr.com/photos/gazeronly/8058105980/ by torbakhopper
#JDKIO @aiborisov