GCP, as a full managed TensorFlow training, prediction and data processing environment.

1. GCP TensorFlow
Google Cloud ML Engine & Dataflow
H.Yoshikawa (@hayatoy)
TFUG #5
24 May 2017

2. @hayatoy
APAC
TFUG
GCPUG
GDG
Presenter

5. Google Cloud ML Engine
Jupyter Notebook [DEMO]
Online Prediction [DEMO]
Google Cloud Dataflow

6. Cloud ML Engine

7. Cloud ML Engine
TensorFlow
Online Prediction
←ktkr!

8. Cloud ML Engine

9. Cloud ML Engine

11. Package
gcloud
REST API
Package
Jupyter Notebook ←New!

12. gcloud
GCP
gcloud ml-engine jobs submit
COMMAND [GLOBAL-FLAG …]

13. REST API
Python Client Library
from oauth2client.client import GoogleCredentials
from googleapiclient import discovery
credentials = GoogleCredentials.get_application_default()
ml = discovery.build('ml', 'v1', credentials=credentials)
job_req = ml.projects().jobs().create(
parent=project_id,
body=param)
job_req.execute()

14. Jupyter Notebook
Notebook Cloud ML Engine
https://github.com/hayatoy/cloudml‑magic

16. Online Prediction ‑ Demo

17. pix2pix GCPUG
bit.ly/realgcpug

18. pix2pix

19. GCPUG
Original Edge 20 epochs 200 epochs

20. Online Prediction
SavedModel v1
serving_input_fn
Beta

21. Cloud Dataflow

22. Cloud Dataflow
ETL
tf.Transform
Dynamic Work Rebalancing
GCE
MapReduce

23. Dataflow

24. Dataflow

25. BigQuery, GCS
query = "SELECT * FROM [project:dataset.table]"
p = beam.Pipeline(options=options)
(p | 'query image list' >> beam.Read(beam.io.BigQuerySource(query=query))
| 'download image' >> beam.FlatMap(download_image)
| 'crop and resize image' >> beam.Map(crop_image)
| 'make edge image' >> beam.Map(make_edge_image)

26. 1000
DWR
Cap
opt.max_num_workers = 10

27. Dataflow

29. GCP

30. Thank you!
Cloud ML Engine
https://cloud.google.com/ml‑engine/
Cloud Dataflow
https://cloud.google.com/dataflow/
Cloud ML Engine ‑ Jupyter Notebook Extension
https://github.com/hayatoy/cloudml‑magic