Introduction to Google Cloud data analysis and machine learning technologies, including DialogFlow and ML APIs.

1. Google Data Analysis
Technologies
Andres L. Martinez a.k.a almo
Google Developer Relations Manager
@davilagrau

2. Contents
● BigQuery
● Data Analysis
● Machine
Learning
● Cloud ML
● Google Assistant
● Cloud ML APIs

3. BigQuery

4. BigQuery: 100% serverless data warehouse
Google
BigQuery
Fully Managed and Serverless
Google Cloud’s Enterprise Data
Warehouse for Analytics
Petabyte-Scale and Fast
Convenience of SQL
Encrypted, Durable and Highly
Available

5. BigQuery is a great choice because:
Near-real
time analysis
of massive
datasets
No-ops;
Pay for use
Durable
(replicated),
inexpensive
storage
Immutable
audit logs
Mashing up
different
datasets to
derive insights

6. 10 B rows
Sample query - Processes over 10
billion rows in less than 10
seconds
SELECT
language, SUM(views) as views
FROM
wikipedia_benchmark.Wiki10B
WHERE
regexp_match(title,"G.*o.*o.*g")
GROUP by language
ORDER by views DESC

7. BigQuery = Massively Parallel Processing query
with the petabit network and thousands of servers
SQL QueryPetabit Network
BigQuery
Storage Compute
Streaming Ingest
Fast Batch Load
DataFlow
DataProc

8. Load data using bq tool, web UI, or API
Create, append or
overwrite table
CSV, JSON or
AVRO format

9. Data Studio

10. For business analysts
Beautiful reports
Drive-based collaboration experience
No technical expertise required
Connects to many sources: BigQuery, Adwords, Google
Analytics, Google Sheets, YouTube Analytics, etc.

11. Integrating with Google Data Studio
1 Navigate to DataStudio
to create a new
dashboard
2 Create a new Data
Source
3 Select the type of Data
Source to use
4 Authorize

12. Person
A
Person
B
Collaboration

13. Sample Dashboard

14. Machine Learning

15. Why Machine Learning?
★ Allows to solve problems we don’t have exact solution
for.
○ E.g. recommendations, predictions, clustering.
★ Given y = F(X), where we observe y, we can estimate F.
★ Becomes better with more data
○ when hard coded solution usually becomes worse with more code :)

16. Google Products Using Machine Learning

17. Big Data Challenges
★ Variety of data
★ Learning many things at once
★ Small data where matters

18. Why is this time different?Photo by Emily Morter on Unsplash

19. Photo by Sharon Pittaway on Unsplash
Innovation

20. Photo by Alex Holyoake on Unsplash

21. Fast{er}
Photo by Josh Calabrese on Unsplash

22. Focus on the user

23. Deep Learning
Input
Out

24. Input
Out
Deep Learning for Perception Tasks

25. Input
Out
Deep Learning for Perception Tasks

26. Input
Out
Deep Learning for Perception Tasks

27. Input
Out
Deep Learning for Perception Tasks

28. Input
Out
Deep Learning for Perception Tasks

29. Deep Learning combines many components
Predictions
GoogLeNet
Recurrent Neural Network

30. The Machine Learning Spectrum
TensorFlow Cloud Machine Learning Machine Learning APIs
BYOML skills
(Friendly Machine Learning)
Pre-packaged ML

34. TensorFlow Ecosystem
Researchers Developers
Data
Scientists

35. TensorFlow Core Execution Engine
CPU GPU Android iOS ...
C++ FrontendPython Frontend ...

36. TensorFlow: Google backed
★ Google supported (growing army of
engineers are working on improving it).
★ Used in 100s of products across Google

37. Simple example:
How can I use this?

38. Predictive Example
Going to deep neural network is easy:

39. Understanding Images

40. Image Classification

42. Scaling Out
TensorFlow scales with number of Machines.
You can use Google Cloud ML or Docker containers in VMs.
https://arxiv.org/abs/1604.00981

43. TensorFlow Serving: Serving models in production
Open Source project.
Check it out:
http://github.com/tenso
rflow/serving

44. Google Assistant

45. Voice Kit

46. Google Home — voice-activated speaker powered
The Google Assistant — A conversation between you and
Google that helps you get more done in your world.
Actions on Google — How developers can extend the
assistant (via Conversation Actions)

47. Direct Actions

48. Conversation Actions

49. Use machine
learning to
understand what
users are saying

51. “Ok Google, talk to personal chef”
Conversation API, Actions SDK
Invoke “personal chef” action
“Sure, here’s personal chef.
Hi, I’m your personal chef, what
are you in the mood for?”
Speech to Text
“What protein would you
like to use?”
“Well, it’s kind of cold outside, so I’d like
something to …”
Text to Speech
“Sure, here’s your personal chef”
Speech to Text, NLP,
Knowledge Graph,
ML Ranking, User
Profile, Text to
Speech

52. APIs

53. Confidential & ProprietaryGoogle Cloud Platform 53
So…. Why APIs?
{ Google Cloud Platform }
1. We want to offer businesses the tools to differentiate by offering a powerful set of APIs
that enable apps to see, hear and understand the world
2. Reduce your Time to Market (TMM) when launching your next-generation app
3. Provide you easy access to machine learning technology to give any developer the
freedom to work in the language and tools they want
4. Provide virtually limitless scalability to your application without needing to manage
back-end servers running deep learning

54. Pre-Trained Machine Learning Models
Fully trained ML models from Google Cloud that allow a general developer to
take advantage of rich machine learning capabilities with simple REST based
services.

55. Introducing
Cloud Natural Language API
Sentiment analysis and entity
recognition for text

56. Confidential & ProprietaryGoogle Cloud Platform 56
Features
Extract sentence, identify parts of
speech and create dependency parse
trees for each sentence
Identify entities and label by types such
as person, organization, location, events,
products and media
Understand the overall sentiment of a
block of text
Access via REST API. Text can be
uploaded in the request or integrated
with Google Cloud Storage
Syntax Analysis Entity Recognition
Sentiment Analysis Integrated REST API

57. Cloud Vision API
Insight from images with our powerful
Cloud Vision API

58. Confidential & ProprietaryGoogle Cloud Platform 58
Faces: Faces, facial landmarks,
emotions
OCR: Read and extract text, with
support for > 10 languages
Photo credit Getty Images
Label: Detect entities from furniture to
transportation
Logos: Identify product logos
Landmarks & Image Properties
Detect landmarks & dominant
color of image
Safe Search: Detect explicit content -
adult, violent, medical and spoof
Cloud Vision API
Call API from anywhere, with support for embeddable images, and Google Cloud Storage

59. Use case

60. Let’s Party

63. Party planning
● Finding people @Twitter
● Cloud Vision API
● Custom classifier
(k-means)

65. Google Cloud Console
We need to have access so we can
add hash tag to intro slide

66. Google Cloud
Console

68. Show Me
the code!

69. Main shellplus_contacts = get_plus_contacts()
print "Processing %d contacts" % len(plus_contacts)
for plus_id in plus_contacts:
plus_profile = get_plus_profile(plus_id)
image_uri = plus_profile['image']['url'].replace("?sz=50","?sz=250")
image_data = analyze_img(image_uri)
if image_data is not None:
print(image_uri)
if 'labelAnnotations' in image_data['responses'][0]:
for label in image_data['responses'][0]['labelAnnotations']:
print label['description']; label['score']; image_uri

70. get_plus_contacts: oAuth
storage = Storage('/home/almo/dev/keys/ex1/oAuth_credentials.dat')
credentials = storage.get()
if credentials is None or credentials.invalid:
PEOPLE_API='https://www.googleapis.com/auth/contacts.readonly'
flow = flow_from_clientsecrets('/home/almo/dev/keys/ex1/oAuth_key.json',
scope=[PEOPLE_API])
credentials = run_flow(flow, storage)
http = credentials.authorize(httplib2.Http())
service = build('people','v1',http=http)
request = service.people().connections().list(resourceName='people/me',
pageSize=500)

71. analyze_image
api_key = json.load(open('/home/almo/dev/keys/ex1/api_key.json'))['api_key']
service = discovery.build('vision','v1',developerKey=api_key)
service_request = service.images().annotate(body={
'requests': [{
'image': {
'content': image_content.decode('UTF-8')
},
'features': [{
'type': 'LABEL_DETECTION',
'maxResults': 3
}]
}]
}

72. Data

73. face; 0,92830354; https://lh3.googleusercontent.com/-c3M1gn6ougg/AAAAAAAAAAI/AAAAAAAAAds/cTIrpGhktfw/photo.jpg?sz=250
text; 0,93046468; https://lh4.googleusercontent.com/-GFVyrVlgMy4/AAAAAAAAAAI/AAAAAAAAABE/u3xVd9eJgf8/photo.jpg?sz=250
font; 0,85384184; https://lh4.googleusercontent.com/-GFVyrVlgMy4/AAAAAAAAAAI/AAAAAAAAABE/u3xVd9eJgf8/photo.jpg?sz=250
line; 0,70535356; https://lh4.googleusercontent.com/-GFVyrVlgMy4/AAAAAAAAAAI/AAAAAAAAABE/u3xVd9eJgf8/photo.jpg?sz=250
eyebrow; 0,98022038; https://lh5.googleusercontent.com/-5c9gdP9nX9M/AAAAAAAAAAI/AAAAAAAAGt4/FoZEEVA8F68/photo.jpg?sz=250
hair; 0,96653992; https://lh5.googleusercontent.com/-5c9gdP9nX9M/AAAAAAAAAAI/AAAAAAAAGt4/FoZEEVA8F68/photo.jpg?sz=250
face; 0,95101357; https://lh5.googleusercontent.com/-5c9gdP9nX9M/AAAAAAAAAAI/AAAAAAAAGt4/FoZEEVA8F68/photo.jpg?sz=250
person; 0,92170084; https://lh4.googleusercontent.com/-yVWpXcqQfXU/AAAAAAAAAAI/AAAAAAAAB5w/rqxRrJHgk_0/photo.jpg?sz=250
news; 0,63342041; https://lh4.googleusercontent.com/-yVWpXcqQfXU/AAAAAAAAAAI/AAAAAAAAB5w/rqxRrJHgk_0/photo.jpg?sz=250
professional; 0,61274487;
https://lh4.googleusercontent.com/-yVWpXcqQfXU/AAAAAAAAAAI/AAAAAAAAB5w/rqxRrJHgk_0/photo.jpg?sz=250
drawer; 0,80023241; https://lh6.googleusercontent.com/-Qf9SSsIUktA/AAAAAAAAAAI/AAAAAAAAABg/u6zPUNXCYFs/photo.jpg?sz=250
furniture; 0,79278195; https://lh6.googleusercontent.com/-Qf9SSsIUktA/AAAAAAAAAAI/AAAAAAAAABg/u6zPUNXCYFs/photo.jpg?sz=250
product; 0,76023591; https://lh6.googleusercontent.com/-Qf9SSsIUktA/AAAAAAAAAAI/AAAAAAAAABg/u6zPUNXCYFs/photo.jpg?sz=250
eyewear; 0,97702742; https://lh3.googleusercontent.com/-ihQNk3ewmzQ/AAAAAAAAAAI/AAAAAAAAAMk/EEEylEyriNE/photo.jpg?sz=250
hair; 0,96766639; https://lh3.googleusercontent.com/-ihQNk3ewmzQ/AAAAAAAAAAI/AAAAAAAAAMk/EEEylEyriNE/photo.jpg?sz=250
sunglasses; 0,96445274; https://lh3.googleusercontent.com/-ihQNk3ewmzQ/AAAAAAAAAAI/AAAAAAAAAMk/EEEylEyriNE/photo.jpg?sz=250
person; 0,92747426; https://lh4.googleusercontent.com/--_BxhkQPYfA/AAAAAAAAAAI/AAAAAAAAACA/1pN6-Chy8EI/photo.jpg?sz=250
person; 0,96007371; https://lh3.googleusercontent.com/-sX8l_lv_-7w/AAAAAAAAAAI/AAAAAAAAAPU/ApQpBMPbcdc/photo.jpg?sz=250
face; 0,95332307; https://lh3.googleusercontent.com/-sX8l_lv_-7w/AAAAAAAAAAI/AAAAAAAAAPU/ApQpBMPbcdc/photo.jpg?sz=250
Raw Data

74. 160 different labels
Max Freq.: 200
Min Freq. : 1

75. person 200 0,9320951099
hair 140 0,9609928544
face 139 0,9489352931
font 136 0,7606724908
text 130 0,925080287
blue 114 0,9112923658
facial hair 36 0,8802876539
nose 34 0,8859786603
profession 30 0,569073382
hairstyle 25 0,7532089968

76. cartoon 23 0,8588066957
professional 23 0,6149233535
glasses 20 0,8234816515
facial expression 14 0,9502550086
eyebrow 12 0,9559630675
black and white 11 0,9199305709
eyewear 11 0,9767648145
logo 11 0,7749610755
head 9 0,7432496333
clothing 7 0,9151270214

78. [{}]

80. Planning our next
Party

81. "hair", 0.9559916, "person", 0.94347906, "face", 0.92830354
"text", 0.9304647, "font", 0.85384184, "line", 0.70535356
"eyebrow", 0.9802204, "hair", 0.9665399, "face", 0.9510135
"person", 0.92170084, "news", 0.63342035, "professional", 0.61274487
"drawer", 0.8002325, "furniture", 0.792782, "product", 0.760235
"eyewear", 0.9770274, "hair", 0.9676664, "sunglasses", 0.96445274
"person", 0.92747426,
"https://lh4.googleusercontent.com/--_BxhkQPYfA/AAAAAAAAAAI/AAAAAAAAACA/1pN6-Chy8EI/photo.jpg
?sz=250"
"green", 0.9307698, "text", 0.92834556, "font", 0.8631033
"hair", 0.98155975, "face", 0.95545304, "eyebrow", 0.93590355
"face", 0.9523797, "person", 0.94760686, "hair", 0.94507515
"hair", 0.9731342, "face", 0.94925183, "person", 0.9371813
"hair", 0.94741917, "person", 0.9436425, "hairstyle", 0.7414854
"person", 0.925232, "people", 0.9086431, "male", 0.83032143
"person", 0.95530343, "face", 0.94757956, "nose", 0.86752254
"face", 0.96074444, "hair", 0.9606222, "eyebrow", 0.9451414
"face", 0.9664352, "hair", 0.9561741, "nose", 0.9222636
"phenomenon", 0.94444287, "celestial event", 0.53744316, "aurora", 0.52995497
"face", 0.9625666, "hair", 0.9514838, "facial expression", 0.94977105
"product", 0.80306137, "font", 0.77923214, "logo", 0.69078964
"black and white", 0.9267871, "person", 0.8998944, "photography", 0.8296365

82. Training
mode!

83. “invited”, "hair", 0.9559916, "person", 0.94347906, "face", 0.92830354
“excluded”, "text", 0.9304647, "font", 0.85384184, "line", 0.70535356
“excluded”, "eyebrow", 0.9802204, "hair", 0.9665399, "face", 0.9510135
“invited”, "person", 0.92170084, "news", 0.63342035, "professional",
0.61274487
“excluded”, "drawer", 0.8002325, "furniture", 0.792782, "product", 0.760235
“excluded”, "eyewear", 0.9770274, "hair", 0.9676664, "sunglasses", 0.96445274
“excluded”, "green", 0.9307698, "text", 0.92834556, "font", 0.8631033
“excluded”, "hair", 0.98155975, "face", 0.95545304, "eyebrow", 0.93590355
“invited”, "face", 0.9523797, "person", 0.94760686, "hair", 0.94507515
“invited”, "hair", 0.9731342, "face", 0.94925183, "person", 0.9371813
“invited”, "hair", 0.94741917, "person", 0.9436425, "hairstyle", 0.7414854
“invited”, "person", 0.925232, "people", 0.9086431, "male", 0.83032143
“invited”, "person", 0.95530343, "face", 0.94757956, "nose", 0.86752254
“excluded”, "face", 0.96074444, "hair", 0.9606222, "eyebrow", 0.9451414
“excluded”, "face", 0.9664352, "hair", 0.9561741, "nose", 0.9222636

84. Prediction
Mode!

85. "hair", 0.9559916, "person", 0.94347906, "face", 0.92830354 “invited”
"text", 0.9304647, "font", 0.85384184, "line", 0.70535356 “excluded”
"eyebrow", 0.9802204, "hair", 0.9665399, "face", 0.9510135 “excluded”
"person", 0.92170084, "news", 0.63342035, "professional", 0.61274487 “invited”
"drawer", 0.8002325, "furniture", 0.792782, "product", 0.760235 “excluded”
"eyewear", 0.9770274, "hair", 0.9676664, "sunglasses", 0.96445274 “excluded”
"green", 0.9307698, "text", 0.92834556, "font", 0.8631033 “excluded”
"hair", 0.98155975, "face", 0.95545304, "eyebrow", 0.93590355 “excluded”
"face", 0.9523797, "person", 0.94760686, "hair", 0.94507515 “invited”
"hair", 0.9731342, "face", 0.94925183, "person", 0.9371813 “invited”
"hair", 0.94741917, "person", 0.9436425, "hairstyle", 0.741485 4 “invited”
"person", 0.925232, "people", 0.9086431, "male", 0.83032143 “invited”
"person", 0.95530343, "face", 0.94757956, "nose", 0.86752254 “invited”
"face", 0.96074444, "hair", 0.9606222, "eyebrow", 0.9451414 “excluded”
"face", 0.9664352, "hair", 0.9561741, "nose", 0.9222636 “excluded”

88. Beyond AI

89. Machine Learning find new
ways for our data center to
save energy

90. Global Fishing
Watch (GFW)

91. A.I. Experiments

92. Progress of AI and Machine Learning
https://goo.gl/FWWBv2

93. Photo by Gerome Viavant on Unsplash
… And the future?

94. Thank you!
Andres L. Martinez a.k.a almo
Google Developer Relations Manager
@davilagrau