Presented at the San Francisco Server-Side Swift Meetup, March 27, 2019.

1. Creating Great
REST and gRPC
API Experiences
(in Swift)
Tim Burks
Google

2. October 4, 2000
CL 23852 by jeff
ProtocolBuffer, a class for encoding hierarchical data in
a compact binary form...
Co-designed and Rev. by Sanjay

10. 10 billion+
API calls
every second

11. Protocol Buffers
a language-neutral, platform-neutral, extensible mechanism for serializing structured data.

12. “Protocol Buffers” means several things
1. A serialization mechanism
2. An interface description language
3. A methodology

13. A Serialization Mechanism
An encoded message is just a stream of bytes
[field_number<<3 + wire_type] [length if necessary] [data]...
$ hexdump /tmp/request.bin
0000000 0a 05 68 65 6c 6c 6f
0a is “0000 1010”, so
field_number = 1 and wire_type = 2

14. An Interface Definition Language
syntax = "proto3";
package echo;
// The message used for echo requests and responses.
message EchoMessage {
string text = 1;
}

15. A Methodology
% protoc echo.proto --swift_out=.
# This runs a plugin called protoc-gen-swift
#
# The plugin generates a Swift source file that implements
# the data structures defined in models.proto and code
# for reading and writing them as serialized bytes.

16. echo.pb.swift
// DO NOT EDIT.
//
// Generated by the Swift generator plugin for the protocol buffer compiler.
// Source: echo.proto
//
// For information on using the generated types, please see the documenation:
// https://github.com/apple/swift-protobuf/
...
/// The message used for echo requests and responses.
struct Echo_EchoMessage {
// SwiftProtobuf.Message conformance is added in an extension below. See the
// `Message` and `Message+*Additions` files in the SwiftProtobuf library for
// methods supported on all messages.
var text: String = String()
var unknownFields = SwiftProtobuf.UnknownStorage()
init() {}
}
// MARK: - Code below here is support for the SwiftProtobuf runtime.

17. A Data Definition Language
syntax = "proto3";
package models;
message User {
string id = 1;
string name = 2;
repeated PlantList plantLists = 3;
}
message PlantList {
string id = 1;
string name = 2;
repeated PlantListItem plants = 11;
}
message PlantListItem {
string id = 1;
string botanical_name = 2;
}

18. Protocol Buffers aren’t just for networking
class UserData {
static let sharedInstance = UserData()
public var user : Models_User
init() {
// read info from UserDefaults
if let userdata = UserDefaults.standard.data(forKey:"user") {
do {
user = try Models_User(serializedData:userdata)
} catch {
user = Models_User()
}
} else {
user = Models_User()
}
}
func save() {
DispatchQueue.main.async {
do {
try UserDefaults.standard.set(self.user.serializedData(), forKey:"user")
} catch (let error) {
print("SAVE ERROR (error)")
}
}
}

19. Interface Builder for Data
message Person {
string name = 1;
int32 id = 2;
string email = 3;
message PhoneNumber {
string number = 1;
}
repeated PhoneNumber phone = 4;
}
Interface Builder: Developers specify
their interfaces using a special tool,
tooling compiles and integrates that into
their apps.
Protocol Buffers: Developers specify
their data structures using a special
language, tooling compiles and
integrates that into their apps.

20. Let’s talk about Networked APIs
From the Google Cloud APIs Design Guide:
“Application Programming Interfaces that operate across a network of
computers. They communicate using network protocols including HTTP,
and are frequently produced by different organizations than the ones that
consume them.”

21. API Styles
1. Remote Procedure Call (RPC)
2. Representational State Transfer (REST)

22. An RPC is just a function call.
service Echo {
// Immediately returns an echo of a request.
rpc Get(EchoRequest) returns (EchoResponse) {}
}

23. What is a REST API?
From the Richardson Maturity Model (as described by Martin Fowler):
Level 3 Hypermedia Controls
Level 2 HTTP Verbs
Level 1 Resources
Level 0 HTTP

24. Hypermedia Controls???
HATEOAS (Hypertext As The Engine Of Application State)
<appointment>
<slot id = "1234" doctor = "mjones" start = "1400" end = "1450"/>
<patient id = "jsmith"/>
<link rel = "/linkrels/appointment/cancel" uri = "/slots/1234/appointment"/>
<link rel = "/linkrels/appointment/addTest" uri = "/slots/1234/appointment/tests"/>
<link rel = "self" uri = "/slots/1234/appointment"/>
<link rel = "/linkrels/appointment/changeTime" uri = "/doctors/mjones/slots?date=20100104@status=open"/>
<link rel = "/linkrels/appointment/updateContactInfo" uri = "/patients/jsmith/contactInfo"/>
<link rel = "/linkrels/help" uri = "/help/appointment"/>
</appointment>
(source: Martin Fowler)

25. REST
Requirements
- Roy Fielding
● A REST API should not be dependent on any single
communication protocol.
● A REST API should not contain any changes to the
communication protocols aside from filling-out or fixing the
details of underspecified bits of standard protocols.
● A REST API should spend almost all of its descriptive effort
in defining the media type(s) used for representing
resources and driving application state, or in defining
extended relation names and/or hypertext-enabled markup
for existing standard media types.
● A REST API must not define fixed resource names or
hierarchies (an obvious coupling of client and server).
● A REST API should never have “typed” resources that are
significant to the client.
● A REST API should be entered with no prior knowledge
beyond the initial URI (bookmark) and set of standardized
media types that are appropriate for the intended audience
(i.e., expected to be understood by any client that might use
the API).

26. 1. HTTP/HTTPS
a. Paths describe resources (nouns)
b. HTTP verbs describe actions
2. JSON Payloads
REST in
Practice

27. OpenAPI
Industry standard format for describing for REST APIs
Originally designed for documentation, now with many other applications: API
Authoring, Validation, Documentation, Analysis, Search, Testing, Mocking,
Management, Code Generation

28. 2010-11: Google API Discovery Format

29. Discovery Format Code Generators

30. github.com/google/google-api-go-client

31. github.com/google/google-api-python-client

32. github.com/google/google-api-swift-client

33. github.com/google/google-auth-library-swift

34. Authorization: How do we tell an API server
that it’s ok to respond to our requests?
We need to provide a token:
var request = URLRequest(url:url)
request.httpMethod = method
request.httpBody = ...
// add any needed request headers
request.setValue(authorization, forHTTPHeaderField:"Authorization")
How do we get that?

35. OAuth2 Authorization Flow
For an implementation in Swift, see BrowserTokenProvider.swift
Client Service
Authorization URL
Sign-In Page (HTML)
Human Confirmation
Browser Redirect w/ Code
Token Request (Code)
Token
Web
Browser

36. Easier ways to get tokens (1 of 2)
If you’re running inside a VM on Google Cloud Platform, you can get a token
from the Google Cloud Metadata Service.
% curl http://metadata/computeMetadata/v1/instance/service-accounts/default/token
{"access_token":"ya29.GqUBUgXcBmIt7vfHsWJT4qVzdhWxwEb2f3tamcA6ykrIsEANZfQnoH0HDCBnlCztLw
cD47w7YENghIucNUIIypLId4C5dXc4H8D93e17MrSbGRe4ipfoQhxPCIhIU3KJsvFjel0HcN2iwf8xURv2z1lWiN
2jkZjzLiMRWPKfSvtBVzuWkIo5uZ5u25IXle3tJ4SICh0-516sU84DFu0wkPO-q1xGpiff","expires_in":179
9,"token_type":"Bearer"}
Then pass “Bearer “ + access_token as the Authorization header.
See GoogleCloudMetadataTokenProvider.swift.

37. Easier ways to get tokens (2 of 2)
If you’re calling a Google API from anywhere, you can use a Service Account.
1. Create and download the account credentials.
2. Create a JWT and sign it with the account credentials.
3. POST the signed JWT to the Google Account Service and get a token!
See ServiceAccountTokenProvider.swift.

38. Aside: Build an HTTP server with swift-server/http

39. ● REST APIs
○ Discovery Format
○ 250+API descriptions
○ 10+ generators
● gRPC APIs
○ Protocol Buffer Language
○ 40+ gRPC-based APIs
○ 7+ target languages

40. gRPC
Open-Source messaging system based on Google’s internal API architecture.
Used for code and documentation generation, API management, and support
services for APIs and microservices running at very large scale.

41. gRPC is owned by the Cloud Native Computing Foundation

42. gRPC Adoption
Microservices: in data centres
Streaming telemetry from network devices
Client Server communication/Internal APIs
Mobile Apps

43. gRPC
Transport Mechanism
Client → Server
Server → Client
Initial
Metadata
MsgMsg
End of
Stream
Status & Trailing
Metadata
Initial
Metadata
MsgMsg Msg
HTTP/2

45. Cross-Language Interoperability
Java
Service
Python
Service
GoLang
Service
C++
Service
gRPC
Service
gRPC
Stub
gRPC
Stub
gRPC
Stub
gRPC
Stub
gRPC
Service
gRPC
Service
gRPC
Service
gRPC
Stub

47. Build a Swift client
and service with
gRPC
https://github.com/grpc/gr
pc-swift/wiki/Try-gRPC-Swi
ft-in-a-Google-Compute-Eng
ine-VM
or just
https://goo.gl/ux4Txh

48. echo.proto
package echo;
service Echo {
// Immediately returns an echo of a request.
rpc Get(EchoRequest) returns (EchoResponse) {}
// Splits a request into words and returns each word in a stream of messages.
rpc Expand(EchoRequest) returns (stream EchoResponse) {}
// Collects a stream of messages and returns them concatenated when the caller closes.
rpc Collect(stream EchoRequest) returns (EchoResponse) {}
// Streams back messages as they are received in an input stream.
rpc Update(stream EchoRequest) returns (stream EchoResponse) {}
}
message EchoRequest {
// The text of a message to be echoed.
string text = 1;
}
message EchoResponse {
// The text of an echo response.
string text = 1;
}

49. main.swift (1/3)
// Unary
if client == "get" {
var requestMessage = Echo_EchoRequest()
requestMessage.text = message
print("Sending: " + requestMessage.text)
let responseMessage = try service.get(requestMessage)
print("get received: " + responseMessage.text)
}
// Server streaming
if client == "expand" {
var requestMessage = Echo_EchoRequest()
requestMessage.text = message
print("Sending: " + requestMessage.text)
let expandCall = try service.expand(requestMessage) {result in }
var running = true
while running {
do {
let responseMessage = try expandCall.receive()
print("Received: (responseMessage.text)")
} catch Echo_EchoClientError.endOfStream {
print("expand closed")
running = false
}
}
}

50. main.swift (2/3)
// Client streaming
if client == "collect" {
let collectCall = try service.collect() {result in }
let parts = message.components(separatedBy:" ")
for part in parts {
var requestMessage = Echo_EchoRequest()
requestMessage.text = part
print("Sending: " + part)
try collectCall.send(requestMessage) {error in print(error)}
sleep(1)
}
let responseMessage = try collectCall.closeAndReceive()
print("Received: (responseMessage.text)")
}

51. main.swift (3/3)
// Bidirectional streaming
if client == "update" {
let updateCall = try service.update() {result in}
DispatchQueue.global().async {
var running = true
while running {
do {
let responseMessage = try updateCall.receive()
print("Received: (responseMessage.text)")
} catch Echo_EchoClientError.endOfStream {
print("update closed")
latch.signal()
break
} catch (let error) {
print("error: (error)")
}
}
}
...
...
let parts = message.components(separatedBy:" ")
for part in parts {
var requestMessage = Echo_EchoRequest()
requestMessage.text = part
print("Sending: " + requestMessage.text)
try updateCall.send(requestMessage) {error in print(error)}
sleep(1)
}
try updateCall.closeSend()
// Wait for the call to complete.
latch.wait()
}
}

52. EchoService.swift (1/3)
class EchoProvider : Echo_EchoProvider {
// get returns requests as they were received.
func get(request : Echo_EchoRequest, session : Echo_EchoGetSession) throws -> Echo_EchoResponse {
var response = Echo_EchoResponse()
response.text = "Swift echo get: " + request.text
return response
}
// expand splits a request into words and returns each word in a separate message.
func expand(request : Echo_EchoRequest, session : Echo_EchoExpandSession) throws -> Void {
let parts = request.text.components(separatedBy: " ")
var i = 0
for part in parts {
var response = Echo_EchoResponse()
response.text = "Swift echo expand ((i)): (part)"
try session.send(response)
i += 1
sleep(1)
}
}

53. EchoService.swift (2/3)
// collect collects a sequence of messages and returns them concatenated when the caller closes.
func collect(session : Echo_EchoCollectSession) throws -> Void {
var parts : [String] = []
while true {
do {
let request = try session.receive()
parts.append(request.text)
} catch Echo_EchoServerError.endOfStream {
break
} catch (let error) {
print("(error)")
}
}
var response = Echo_EchoResponse()
response.text = "Swift echo collect: " + parts.joined(separator: " ")
try session.sendAndClose(response)
}

54. EchoService.swift (3/3)
// update streams back messages as they are received in an input stream.
func update(session : Echo_EchoUpdateSession) throws -> Void {
var count = 0
while true {
do {
let request = try session.receive()
count += 1
var response = Echo_EchoResponse()
response.text = "Swift echo update ((count)): (request.text)"
try session.send(response)
} catch Echo_EchoServerError.endOfStream {
break
} catch (let error) {
print("(error)")
}
}
try session.close()
}
}

55. Why generate client libraries for APIs?
Make it easier to use Google APIs.
● Hide implementation details from API consumers.
● Develop API clients faster.
● Improve API client quality.
○ Better performance
○ Better consistency
○ Fewer bugs

56. Generated API Clients
a.k.a.
“GAPICs”

57. GAPIC Customers
Google Cloud APIs
Google Ads APIs
…other Google teams
…external API producers?

58. GAPIC Features and Benefits
● Usability:
○ Trivial getting-started experience
○ Flattening
○ Natural paging
○ Long-running operations
○ Sample code for every method
○ Resource name types (e.g. for /projects/{project}/topics/{topic})
● Performance:
○ Batching
● Reliability:
○ Automatic retries

59. Trivial Getting-Started Experience
GAPIC client:
Channel channel =
NettyChannelBuilder.forAddress(API_ENTRY, API_PORT).build();
List<ClientInterceptor> interceptors = new ArrayList<>();
GoogleCredentials credentials =
GoogleCredentials.getApplicationDefault();
interceptors.add(
ChannelFactory.credentialInterceptor(credentials));
LoggingService stub =
LoggingServiceGrpc.newBlockingStub(channel, interceptors);
gRPC:
LoggingClient client = LoggingClient.create();

60. Automatic Retry and Flattening
GAPIC client:
LoggingService stub = ...;
// Note: this only does a simple retry, exponential backoff
// is more complex
DeleteLogRequest request = DeleteLogRequest.newBuilder()
.setLogName(LOG_NAME)
.build();
for (int i = 0; i < MAX_RETRY; i++) {
try {
stub.deleteLog(request);
} catch (RpcException e) {
if (i == MAX_RETRY) throw e;
}
}
gRPC:
LoggingClient client = LoggingClient.create();
client.deleteLog(LOG_NAME);

61. Natural Pagination
GAPIC client:
LoggingService stub = ...;
ListLogsRequest request = ListLogsRequest.newBuilder()
.setParentName("projects/" + PROJECT_ID)
.build();
while (true) {
ListLogsResponse response = stub.listLogs(request);
for (Log log : response.getLogsList()) {
System.out.printf("Log: %s%n", log.getDisplayName());
}
String nextPageToken = response.getNextPageToken();
if (nextPageToken != null) {
request = ListLogsRequest.newBuilder()
.setPageToken(nextPageToken).build();
} else {
break;
}
}
gRPC:
LoggingClient client = LoggingClient.create();
ParentNameOneOf parent =
ParentNameOneOf.from(ProjectName.create(PROJECT_ID));
for (Log log : client.listLogs(PARENT).iterateAll()) {
System.out.printf("Log: %s%n", log.getDisplayName());
}

62. Long-Running Operations
SpeechClient speechClient = SpeechClient.create();
OperationFuture<LongRunningRecognizeResponse> recognizeOperation =
speechClient.longRunningRecognizeAsync(config, audio);
…
LongRunningRecognizeResponse response = recognizeOperation.get();
● Java: OperationFuture
○ Polls the service until the LRO is done
○ Provides metadata as the LRO is in progress
○ Provides the final result

63. Resource Name Types
GAPIC client:
PublisherService stub = ...;
CreateTopicRequest request = CreateTopicRequest.newBuilder()
.setTopic("projects/my-project/topics/my-topic")
.build();
Topic response = stub.createTopic(request);
gRPC:
TopicAdminClient topicAdminClient = TopicAdminClient.create();
TopicName name = TopicName.create("my-project", "my-topic");
Topic response = topicAdminClient.createTopic(name);

64. Generated Sample Code

65. GAPIC Project Architecture (hosted at github.com/googleapis)
gapic-generator-python
WANTED: ruby, swift, rust, elixir...
gapic-generator
(java, php, python, go,
csharp, node, ruby)
gapic-generator-kotlin
gapic-generator-go
gapic-generator-csharp
gapic-generator-typescript
One API
description
(Annotated
.proto files)
Many
language-
specific
client
libraries

66. Standard (Language-Independent) API Client Configuration

67. “Showcase” API verification

68. What can else we get from an API description format?
● Generated CLIs
● Generated Documentation
● API review and governance
● API management services
● Server frameworks and tools
● Generated mock servers
● Automatic testing
● API search and discovery

69. Manage Your Service With Endpoints
Develop, deploy and manage APIs on any Google Cloud Platform backend.
● User Authentication via JSON Web Token validation
● Logging and Monitoring
● API Keys
● Easy integration + deployment

70. Endpoints Architecture
GCE, GKE, Kubernetes or App Engine
Flexible
Environment Instance
GCE, GKE, Kubernetes or App Engine Flexible
Environment Instance (or off-GCP)
Extensible Service
Proxy Container
API Container
Google Service
Management API
User
Code
api
calls
gcloud
Config
deployment
Google Cloud
Console
Google Service
Control API
config Runtime
check & report
Load
balanci
ng
Stackdriver
Metrics &
logs
Metrics &
logs
visualized

71. Generate FileDescriptorSet
protoc kiosk.proto
-I api-common-protos
-I .
--include_imports
--include_source_info
--descriptor_set_out=generated/kiosk_descriptor.pb
--go_out=plugins=grpc:generated

72. Describing the Service
type: google.api.Service
config_version: 3
name: my-api.endpoints.my-gcp-project.cloud.goog
title: Kiosk gRPC API
apis:
- name: kiosk.Display

73. Starting Endpoints Proxy in Front of Application
gcloud endpoints services deploy service.yaml kiosk_descriptor.pb
./endpoints/START_ENDPOINTS_PROXY.sh my-gcp-project my-api <keyfile>

74. $ export KIOSK_PORT=8083
$ k list kiosks
FROM localhost:8083 LIST KIOSKS
rpc error: code = Unauthenticated
desc = Method doesn't allow
unregistered callers (callers
without established identity).
Please use API Key or other form of
API consumer identity to call this
API.
Starting Endpoints Proxy in Front of Application

75. $ export KIOSK_APIKEY=AIzaSy[...]bBo
$ k list kiosks
FROM localhost:8083 2018/09/21 19:08:25 Using API key: AIzaSy[...]bBo
LIST KIOSKS
id:1 name:"HTTP Kiosk" size:<width:1080 height:720 >
id:2 name:"HTTP Kiosk" size:<width:1080 height:720 >
Getting an API Key

76. Build a gRPC service, get a free REST API!

77. service Display {
rpc CreateKiosk(Kiosk) returns (Kiosk) {
option (google.api.http) = {post:"/v1/kiosks"};
}
rpc GetKiosk(GetKioskRequest) returns (Kiosk) {
option (google.api.http) =
{get: "/v1/kiosks/{id}" };
}
rpc DeleteKiosk(DeleteKioskRequest)
returns (google.protobuf.Empty) {
option (google.api.http) =
{ delete: "/v1/signs/{id}" };
}
[...]
}
Add REST annotations to your service
gRPC definitions with HTTP annotations
service Display {
rpc CreateKiosk(Kiosk) returns (Kiosk) {
}
rpc GetKiosk(GetKioskRequest) returns(Kiosk) {
}
rpc DeleteKiosk(DeleteKioskRequest)
returns (google.protobuf.Empty) {
}
[...]
}
gRPC definitions

78. # Add a kiosk
$ curl -d '{"name":"HTTP Kiosk", "size": { width: 1080, height: 720 } }'
localhost:8082/v1/kiosks?key=AIzaSy[...]bBo
# Get kiosk
$ curl localhost:8082/v1/kiosks/1?key=AIzaSy[...]bBo
Now you can use HTTP+JSON!

79. Use Endpoints, get a free developer portal!

80. Build GraphQL interfaces to gRPC services
with rejoiner

81. BONUS: Call Google RPC APIs with HTTP/1 using gRPC Fallback

82. We’re in the Summer of Code!

83. Have a Slice of APIs with us!

84. And don’t miss: Swift for Tensorflow