A quick tour of various services on Google Cloud Platform for data scientists/engineers.

1. Ta Virot Chiraphadhanakul, PhD (@tvirot)
GDE in Machine Learning | Managing Director @ Skooldio
Data Science on Google Cloud Platform
Google Developers
Launchpad Build for Cloud Meetup, Bangkok

2. 90% of the data in the world
today has been created in
the last two years alone.
— IBM

3. Turning data into …
ǡ Metrics
Insights
Data Products
@tvirot

4. Data Science Process

Collect Manipulate Analyze Model
|
Communicate
@tvirot

5. A lot of tools needed
Pig Airflow
@tvirot

6. Administrative and operational issues
• Deployment and configuration
• Managing scale and optimizing utilization
• Reliability
• Resource provisioning
• etc.

7. @tvirot

8. How about a serverless big
data stack that scales
automatically?

9. Serverless Data Processing
• Focus on insights, not administration
• Practically infinite scale, exactly when you need it
• Pay only for what you use
• Freedom to experiment, fail quickly, and iterate. Successful experiments are
ready to go live right away

10. Storage & Databases Big Data Machine Learning
Data Science on Google Cloud Platform

11. 
Collect Manipulate Analyze Model
|
Communicate

12. Storage & Databases
Cloud Storage
A scalable object storage service
suitable for all kinds of
unstructured data
Cloud SQL
A fully-managed database service
that makes it easy to set up,
maintain, manage, and administer
your relational MySQL and
PostgreSQL databases in the cloud
Cloud Datastore
A highly-scalable NoSQL database
for your applications. Cloud
Datastore automatically handles
sharding and replication.
Cloud BigTable
A massively scalable NoSQL
database suitable for low-latency
and high-throughput workloads. It
supports the open-source, industry-
standard HBase API

13. Fully-managed real-time messaging service
that allows you to send and receive
messages between independent applications
Connect Anything to Everything
Use Cloud Pub/Sub to publish and
subscribe to data from multiple sources,
reducing dependencies between
components of distributed applications
Highly Scalable
Any customer can send up to 10,000
messages per second, by default
Guaranteed Delivery
Designed to provide “at least once” delivery
Cloud Pub/Sub

14. 
Collect Manipulate Analyze Model
|
Communicate

15. Fully-managed data processing service,
supporting both stream and batch execution
of pipelines
Fully Managed
Dynamically provision resources to
minimize latency while maintaining high
utilization efficiency
Unified Programming Model
Express computational requirements
regardless of data source
Cloud Dataflow

16. https://cloud.google.com/dataflow/examples/wordcount-example
Pipeline p = Pipeline.create(options);

17. Pipeline p = Pipeline.create(options);
p.apply(TextIO.Read.from(“gs://dataflow-samples/shakespeare/kinglear.txt”))
Create a Pipeline
https://cloud.google.com/dataflow/examples/wordcount-example

18. Pipeline p = Pipeline.create(options);
p.apply(TextIO.Read.from(“gs://dataflow-samples/shakespeare/kinglear.txt”))
.apply(ParDo.named("ExtractWords").of(new DoFn<String, String>() {
@Override
public void processElement(ProcessContext c) {
for (String word : c.element().split("[^a-zA-Z']+")) {
if (!word.isEmpty()) {
c.output(word);
}
}
}
}))
Read lines
Create a Pipeline
https://cloud.google.com/dataflow/examples/wordcount-example

19. Pipeline p = Pipeline.create(options);
p.apply(TextIO.Read.from(“gs://dataflow-samples/shakespeare/kinglear.txt”))
.apply(ParDo.named("ExtractWords").of(new DoFn<String, String>() {
@Override
public void processElement(ProcessContext c) {
for (String word : c.element().split("[^a-zA-Z']+")) {
if (!word.isEmpty()) {
c.output(word);
}
}
}
}))
.apply(Count.<String>perElement())
Read lines
Create a Pipeline
Tokenize lines into words
https://cloud.google.com/dataflow/examples/wordcount-example

20. Pipeline p = Pipeline.create(options);
p.apply(TextIO.Read.from(“gs://dataflow-samples/shakespeare/kinglear.txt”))
.apply(ParDo.named("ExtractWords").of(new DoFn<String, String>() {
@Override
public void processElement(ProcessContext c) {
for (String word : c.element().split("[^a-zA-Z']+")) {
if (!word.isEmpty()) {
c.output(word);
}
}
}
}))
.apply(Count.<String>perElement())
.apply(MapElements.via(new SimpleFunction<KV<String, Long>, String>() {
@Override
public String apply(KV<String, Long> element) {
return element.getKey() + ": " + element.getValue();
}
}))
Read lines
Create a Pipeline
Tokenize lines into words
Count words
https://cloud.google.com/dataflow/examples/wordcount-example

21. Pipeline p = Pipeline.create(options);
p.apply(TextIO.Read.from(“gs://dataflow-samples/shakespeare/kinglear.txt”))
.apply(ParDo.named("ExtractWords").of(new DoFn<String, String>() {
@Override
public void processElement(ProcessContext c) {
for (String word : c.element().split("[^a-zA-Z']+")) {
if (!word.isEmpty()) {
c.output(word);
}
}
}
}))
.apply(Count.<String>perElement())
.apply(MapElements.via(new SimpleFunction<KV<String, Long>, String>() {
@Override
public String apply(KV<String, Long> element) {
return element.getKey() + ": " + element.getValue();
}
}))
.apply(TextIO.Write.to("gs://my-bucket/counts.txt"));
Format strings
Read lines
Create a Pipeline
Tokenize lines into words
Count words
https://cloud.google.com/dataflow/examples/wordcount-example

22. Pipeline p = Pipeline.create(options);
p.apply(TextIO.Read.from(“gs://dataflow-samples/shakespeare/kinglear.txt”))
.apply(ParDo.named("ExtractWords").of(new DoFn<String, String>() {
@Override
public void processElement(ProcessContext c) {
for (String word : c.element().split("[^a-zA-Z']+")) {
if (!word.isEmpty()) {
c.output(word);
}
}
}
}))
.apply(Count.<String>perElement())
.apply(MapElements.via(new SimpleFunction<KV<String, Long>, String>() {
@Override
public String apply(KV<String, Long> element) {
return element.getKey() + ": " + element.getValue();
}
}))
.apply(TextIO.Write.to("gs://my-bucket/counts.txt"));
Format strings
Read lines
Create a Pipeline
Tokenize lines into words
Write to file
Count words
https://cloud.google.com/dataflow/examples/wordcount-example

23. https://cloud.google.com/dataflow/pipelines/design-principles
Different Pipeline Shapes
MULTIPLE TRANSFORMS
+ MERGING
JOINING MULTIPLE
SOURCES

24. Events, metrics
Stream
Batch
Raw logs,
databases, etc.
Cloud Pub/Sub
Cloud Storage
Cloud Dataflow

25. Managed Spark and Hadoop service which is
fast, easy to use, and low cost
Fast & Scalable Data Processing
Create a cluster in minutes and resize them
at any time
Affordable Pricing
Based on actual use, measured by the
minute
Open Source Ecosystem
Move existing projects or ETL pipelines
without redevelopment
Cloud Dataproc

26. Cloud Dataproc
@tvirot

27. Events, metrics
Stream
Batch
Raw logs,
databases, etc.
Cloud Pub/Sub
Cloud Storage
Cloud Dataflow
Batch Cloud Dataproc

28. An intelligent data service for visually
exploring, cleaning, and preparing data
Visually explore data
Intelligent data manipulation
Serverless and works at any scale
Cloud Dataprep

29. https://cloud.google.com/dataprep/

30. Google's fully managed, petabyte scale, low
cost enterprise data warehouse for analytics
Fully Managed
No infrastructure to manage, and you don't
need a database administrator
Speed & Scale
Scans TB in seconds and PB in minutes
Convenience of SQL
Makes it more accessible
Security & Reliability
Automatically encrypts and replicates data
BigQuery

31. Google's fully managed, petabyte scale, low
cost enterprise data warehouse for analytics
Flexible Data Ingestion
Load your data from Google Cloud Storage
or Google Cloud Datastore, or stream it
Fully Integrated
With other Google Cloud products and
third-party applications
BigQuery

32. https://cloud.google.com/blog/big-data/2016/01/anatomy-of-a-bigquery-query
How fast is BigQuery really?

33. BigQuery
Events, metrics
Stream
Batch
Raw logs,
databases, etc.
Cloud Pub/Sub
Cloud Storage
Cloud Dataflow
Batch Cloud Dataproc

34. 
Collect Manipulate Analyze Model
|
Communicate

35. An easy to use interactive tool for data
exploration, analysis, visualization and
machine learning
Integrated & Open Source
Built on Jupyter (formerly IPython).
Enables analysis of your data on BigQuery,
ML Engine, Compute Engine, and Cloud
Storage
Cloud Datalab

36. Photo: https://github.com/googledatalab/datalab

37. Turns your data into informative dashboards
and reports that are easy to read, easy to
share, and fully customizable
Put all your data to work
Easily access all the data sources you
need to understand your business and
make better decisions
Build engaging visualizations
Create beautiful charts and graphs that
bring your data to life
Leverage teamwork that works
Share and collaborate in real time. Work
together quickly, from anywhere.
Cloud Data Studio

38. https://datastudio.google.com
@tvirot

39. https://datastudio.google.com
@tvirot

40. 
Collect Manipulate Analyze Model
|
Communicate

41. Artificial Intelligence
is the new electricity.
— Andrew Ng

42. AlphaGo
The first computer program to
beat a professional human Go
player
Photo: Nature

43. Waymo
The Google self-driving car
project became Waymo with a
mission to make it easy and
safe for people and things to
move around
Photo: Waymo

44. Machine Learning engine and APIs
Custom ML modelsPre-trained ML models
Machine Learning
Engine
TensorFlowVision API
Translation
API
Natural Language
API
Speech API Jobs API

45. Google Cloud
Vision API
Understand the content of
images
• Label Detection
• Optical Character Recognition
• Explicit Content Detection
• etc.
+
https://m.me/youpin.city | https://youpin.city/app
@tvirot

46. Google Cloud
Speech API
Convert audio to text by
applying powerful neural
network models in an easy to
use API
@tvirot

47. A managed service that enables you to easily
build machine learning models, that work on
any type of data, of any size
Scalable Service
Managed distributed training infrastructure
that supports CPUs and GPUs
HyperTune
Automatically tuning your hyper
parameters with HyperTune
Deep Learning Capabilities
Supports any TensorFlow models
Cloud ML Engine

48. BigQuery
Events, metrics
Stream
Batch
Raw logs,
databases, etc.
Cloud Pub/Sub
Cloud Storage
Cloud Dataflow
Batch Cloud Dataproc
Cloud Machine
Learning Engine

49. Large-Scale Deep Learning for Intelligent Computer Systems, Jeff Dean, WSDM 2016

50. http://playground.tensorflow.org/

51. http://playground.tensorflow.org/

52. http://playground.tensorflow.org/

53. TensorKart
Self-driving MarioKart with
TensorFlow
http://kevinhughes.ca/blog/tensor-kart

54. Cucumber Sorter
"Farmers want to focus and
spend their time on growing
delicious vegetables.”
— Makoto Koike
Photos: Google Cloud Platform / Kaz Sato

55. https://codelabs.developers.google.com/codelabs/cloud-tensorflow-mnist

56. https://codelabs.developers.google.com/codelabs/cloud-tensorflow-mnist

57. Serverless
Less ops and administration
No waiting
Queries that used to take hours or days
now take minutes or seconds
Machine Intelligence
Gives everyone access to the deep learning
systems

59. Thank you!
Ta Virot Chiraphadhanakul, PhD (@tvirot)
GDE in Machine Learning | Managing Director @ Skooldio