GraphQL is a query language for APIs and a runtime to fulfill these queries, allowing applications to easily connect and access data stored on any type of database technology or API. AWS AppSync provides a powerful and flexible serverless GraphQL API that securely accesses, manipulates, and combines data from multiple sources at any scale, enabling you to build any kind of application on a range of data sources independently of the underlying database technology. In this session, we discuss different use cases where AWS AppSync and GraphQL power next-generation applications. Special guest, Candid Partners, shares how it uses AWS AppSync in its Data Fabric solution to simplify large-scale data management using a GraphQL API to interact with data lakes.

1. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Next generation intelligent data lakes,
powered by GraphQL & AWS AppSync
Ed Lima
Sr. Solutions Architect
AWS AppSync & AWS Amplify
Amazon Web Services
M A D 3 0 9
Aaron Bawcom
Chief Architect
Candid Partners

2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Agenda
• Data and APIs
• GraphQL: A query language for APIs
• AWS AppSync: Use cases and capabilities
• Data lakes, GraphQL, and AWS AppSync

3. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

4. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
“You can have data without information, but you
cannot have information without data.”
Daniel Keys Moran
“Data really powers everything that we do.”
Jeff Weiner

5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
“A one-size-fits-all database
doesnʼt fit anyone.”
Werner Vogels

6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AWS databases and analytics
Broad and deep portfolio, built for builders
AWS Marketplace
Amazon Redshift
Data warehousing
Amazon EMR
Hadoop + Spark
Amazon Athena
Interactive analytics
Amazon Kinesis
Data Analytics Real
time
Amazon ES
Operational analytics
Amazon RDS
MySQL, PostgreSQL, MariaDB,
Oracle, SQL Server
Amazon Aurora
MySQL, PostgreSQL
Amazon
QuickSight
Amazon
SageMaker
Amazon DynamoDB
Key value, document
Amazon ElastiCache
Redis, Memcached
Amazon Neptune
Graph
Amazon Timestream
Time series
Amazon QLDB
Ledger database
Amazon S3/
Amazon S3 Glacier
AWS Glue
ETL and Data Catalog
AWS Lake Formation
Data lakes
AWS DMS | AWS Snowball | AWS Snowmobile | Amazon Kinesis Data Firehose | Amazon Kinesis Data Streams | AWS Data Pipeline | AWS Direct Connect
Data movement
AnalyticsDatabases
Business intelligence and machine learning
Data lake
Amazon
Managed
Blockchain
AWS Blockchain
Templates
Blockchain
Amazon
Comprehend
Amazon
Rekognition
Amazon
Lex
Amazon
Transcribe
AWS DeepLens 250+ solutions
730+ Database
solutions
600+ Analytics
solutions
25+ Blockchain
solutions
20+ Data lake
solutions
30+ solutions
Amazon RDS
on VMWare

7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T

8. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
InternetMobile/web
apps
Databases/
data stores
Basic API technology stack
Failover
Load balancers
Web/application servers
Message buses
Workers
?API backend
?API “server”

10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Basic API technology stack
Databases/
data stores
API backendAPI “server”

11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
InternetMobile/web
apps
Databases/
data stores
Basic API technology stack
?API backend
?API “server”
AWS
Fail over
Load balancers
Web/application servers
Message buses
Workers

12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
No servers to provision
or manage
Scales with usage
Never pay for idle Availability and fault
tolerance built in
Serverless means…

13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T

14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
InternetMobile/web
apps
Basic serverless API technology stack
AWS
Serverless
business logic
Serverless
APIs
Databases/
data stores
Fail over
Load balancers
Web/application servers
Message buses
Workers

15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
/posts /comments /authors
REST API
posts comments authors
GraphQL API
What is GraphQL?

16. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
https://2018.stateofjs.com/data-layer

17. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
https://2018.stateofjs.com/data-layer

18. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
https://2018.stateofjs.com/data-layer

19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Queries MutationsTypes
Subscriptions
GraphQL schema and operations
A query language for APIs—and a runtime!
type User {
id: ID!
username: String!
firstName: String
lastName: String
daysActive: Int
}

20. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
A query language for APIs
Queries
query GetPost {
getPost(id: ”1”) {
id
title
}
}
mutation CreatePost {
createPost(title: “Summit”) {
id
title
}
}
subscription OnCreatePost {
onCreatePost {
id
title
}
}
Mutations Subscriptions

21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T

22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AWS AppSync
Managed serverless
GraphQL service
Connect to data sources in
your account
Add data sync, real-time, and offline
capabilities for any data source or
API
GraphQL facade for any AWS
service
Conflict detection and
resolution in the cloud
Enterprise security features:
IAM, Amazon Cognito, OIDC,
API keys

23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Who’s using AWS AppSync in production?

24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
How does AWS AppSync work?
,
,

25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
How does AWS AppSync work?
,
,

26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
How does AWS AppSync work?
,
,

27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
How does AWS AppSync work?
,
,

28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
How does AWS AppSync work?
,
,

29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
GraphQL data flow in AWS AppSync: Resolvers
Request template Response template
Data sources
1 2 3 4 5

30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Pipeline resolvers

31. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Multi-auth
Type, operation, and field-level
authorization
Default authorization mode
Additional providers:
• OpenID Connect
• Amazon Cognito user pools
• AWS IAM
• API key

33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Multi-auth
Type, operation, and field-level
authorization
Default authorization mode
Additional providers:
• OpenID Connect
• Amazon Cognito user pools
• AWS IAM
• API key

34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Log insights

35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Log insights
nanoseconds

36. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Real-time data broadcasting

38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Real-time subscriptions
subscription onCreateMessage {
createMessage {…}
}
WebSocket URL and connection payload
Secure WebSocket connection (wss://)
AppSync Data Sources
AWS AppSync
Amazon
DynamoDB
Amazon Elasticsearch
Service
AWS Lambda
Amazon Aurora
HTTP
Local (Pub/Sub)

39. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

40. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Offline

41. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Chat applications

42. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Internet of Things (IoT)

43. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Microservices data access

44. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Polyglot backend data access

45. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T

46. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
July 2019
Data Fabric

47. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Introduction
Chief Architect, Candid Partners
Aaron Bawcom

48. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Overview
- Terminology
- Data is everywhere
- Data management challenges
- The ideal solution
- Data fabric solution

49. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Terminology
• Data lake
• Data warehouse
• Data mart
• Universal semantic
layer
• Operational data lake

50. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Terminology
• Data provisioning
• Data promotion
• Storage zones
• Data governance

51. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Data is
everywhere
Acquisitions
Business units
Geographies
External partners
Supply chains
Products
Customer engagement
IoT
Websites
Mobile

52. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Data
management
challenges
Most companies only analyze 12%of the data they have
By 2020, every person will generate 1.7 megabytes in just a second
Gartner estimates that poor-quality data is costing organizations on average
$14.2 million annually
“Nearly one third of analysts spend more than 40% of their time vetting and validating their
analytics data before it can be used for strategic decision-making”
Organizations make (often erroneous) assumptions about the state of their data and continue to
experience inefficiencies, excessive costs, compliance risks, and customer satisfaction issues as a result
Corporate data grows 40% per year (Hollis Tibbetts)
Business cost of bad data may be as high as 10% to 25% of an organizations
revenue
40% of businesses lose sales due to inaccessible data

53. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Candid Partners Proprietary & Confidential 54
Asinglesystemtoacquire,store,
progress,refine,andmakeprovisioned
dataaccessibletosystemsofinsight,
dataanalysts,datascientists,and
decisionmakers.
view
Operations
•Helpdesk
•Workflows
•Stewardship
Catalog
•Glossary
•Dictionary
•Lineage
•Search
Universal semantic
layer
•API
•Data warehouse
Acquisition
•Database push/pull
•Streaming
•File
Promotion
•Landing, raw, consolidated,
processed, USL, and data science
Governance
•Policy
•Process
•Security
•Entitlement
The ideal
solution

54. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Data
acquisition
Refinement
Storage Lineage
Data fabric
solution
AccessibilityandsystemIntegration
usingcloud-nativetechnologies

55. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Conceptual pattern
Storage
Configuration
Metadata
Acquisition
Router Processors
Delivery
Userinterface
Efficiently acquire data
Large-scale data
transformation
governance
Bad ETL/ELT codeAPI access

56. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T

57. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Candid Partners Proprietary & Confidential 58
Problem:
Efficiently
acquire data
Acquire data
Cost optimization
Operations
Scale
Flexibility

58. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Candid Partners Proprietary & Confidential 59
Configuration
YAML based with UI
RDBMS push/pull API pull
AWS Lambda, Amazon SQS
Files
AWS Transfer for SFTP, AWS Lambda, Amazon
SQS
Streaming
Amazon API Gateway, Amazon Kinesis, AWS
Lambda, AWS Service Endpoint
Solution:
Continuous data
acquisition

59. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Candid Partners Proprietary & Confidential 60
Problem:
Large-scale data
transformation
governance
Bad source data
Source data systems
Source data systems
ETL dependency challenges

60. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Candid Partners Proprietary & Confidential 61
Solution:
Create simplified
orchestration of data
transformation
Create data router
Amazon DynamoDB, AWS Lambda, Amazon
SQS
Create data processors
Amazon ECS task, AWS Lambda, AWS Glue, Amazon
DynamoDB, Amazon Aurora, Amazon S3, Amazon
Redshift

61. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Candid Partners Proprietary & Confidential 62
Problem:
Bad ETL/ELT code
Data transformation challenges
Difficult to conduct root-cause
analysis
Difficult ETL code engineering
and operations

62. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Candid Partners Proprietary & Confidential 63
Solution:
Create modernized
ETL logic using
AWS AppSync
Automated creation of GraphQL schema
Automated creation of resolvers to cloud-
native storage
Amazon DynamoDB, Amazon S3, Amazon Redshift, Amazon Aurora
Customized creation of resolvers to external
data
Simplified ETL business logic
Simplified debugging
Amazon CloudWatch
Higher-quality ETL code

63. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
GraphQL standardized ETL
rules:
- name: eidrProductUpdate
src: s3:/eidr/product
extract:
query: “query { … }”
variables:
id: $src.productID
transform: |
( { data, src } ) =>
return { …data, …src }
load:
mutate: “mutate (…) { … }”
variables:
input: $transform

64. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Candid Partners Proprietary & Confidential 65
Problem:
No unified API
provided by the
organization
Different suppliers had different
forms of EDI interfaces into the organization
Different portals were provided for different
types of users to order products
Different integrations had different “views”
of product master data
Lack of consistent order execution lead to
penalty fines paid to major customers
No external enterprise authorization and
authentication structure was available from a
consumable API

65. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Candid Partners Proprietary & Confidential 66
Solution:
Create a single unified
external API for e-
commerce
transactions
Continuous data
acquisition
Amazon DynamoDB
Single GraphQL
schema
AWS AppSync

66. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
E-commerce API
type Mutation {
orderProduct(…)
@aws_auth( cognito_groups:
[”corpECommerce"] )
}

67. Thank you!
S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Ed Lima
@ednergizer
Aaron Bawcom
aaron.bawcom@candidpartners.com