The document discusses using Google Cloud Machine Learning APIs to curate online content. It provides examples of analyzing images of people from Google+ contacts to identify labels like "hair", "face", and "person". It then sorts the labels and associated photos into potential guests for a party ("invited") and those that should be excluded. This is done by training a simple model and using it to make predictions about new images.

1. Curating Online Content with
Google ML APIs
Andres L. Martinez a.k.a almo
Google Developer Relations Manager
@davilagrau

2. Resources
Twitter: @davilagrau
GitHub: https://github.com/almo
Linkedin:
https://www.linkedin.com/in/aleonar

3. Unstructured data accounts for
90% of enterprise data*
Cloud Machine Learning APIs
help you make sense of it
*Source: IDC

4. The Machine Learning Spectrum
TensorFlow Cloud Machine Learning Machine Learning APIs
BYOML skills
(Friendly Machine Learning)
Pre-packaged ML

7. Voice Kit

8. Confidential & ProprietaryGoogle Cloud Platform 8
So…. Why APIs?
{ Google Cloud Platform }
1. We want to offer businesses the tools to differentiate by offering a powerful set of APIs
that enable apps to see, hear and understand the world
2. Reduce your Time to Market (TMM) when launching your next-generation app
3. Provide you easy access to machine learning technology to give any developer the
freedom to work in the language and tools they want
4. Provide virtually limitless scalability to your application without needing to manage
back-end servers running deep learning

9. Pre-Trained Machine Learning Models
Fully trained ML models from Google Cloud that allow a general developer to
take advantage of rich machine learning capabilities with simple REST based
services.

10. Introducing
Cloud Natural Language API
Sentiment analysis and entity
recognition for text

11. Confidential & ProprietaryGoogle Cloud Platform 11
Features
Extract sentence, identify parts of
speech and create dependency parse
trees for each sentence
Identify entities and label by types such
as person, organization, location, events,
products and media
Understand the overall sentiment of a
block of text
Access via REST API. Text can be
uploaded in the request or integrated
with Google Cloud Storage
Syntax Analysis Entity Recognition
Sentiment Analysis Integrated REST API

12. Cloud Vision API
Insight from images with our powerful
Cloud Vision API

13. Confidential & ProprietaryGoogle Cloud Platform 13
Faces: Faces, facial landmarks,
emotions
OCR: Read and extract text, with
support for > 10 languages
Photo credit Getty Images
Label: Detect entities from furniture to
transportation
Logos: Identify product logos
Landmarks & Image Properties
Detect landmarks & dominant
color of image
Safe Search: Detect explicit content -
adult, violent, medical and spoof
Cloud Vision API
Call API from anywhere, with support for embeddable images, and Google Cloud Storage

14. Let’s Party

17. Party planning
● Finding people @Twitter
● Cloud Vision API
● Custom classifier
(k-means)

19. Google Cloud Console
We need to have access so we can
add hash tag to intro slide

20. Google Cloud
Console

22. Show Me
the code!

23. Main shellplus_contacts = get_plus_contacts()
print "Processing %d contacts" % len(plus_contacts)
for plus_id in plus_contacts:
plus_profile = get_plus_profile(plus_id)
image_uri = plus_profile['image']['url'].replace("?sz=50","?sz=250")
image_data = analyze_img(image_uri)
if image_data is not None:
print(image_uri)
if 'labelAnnotations' in image_data['responses'][0]:
for label in image_data['responses'][0]['labelAnnotations']:
print label['description']; label['score']; image_uri

24. get_plus_contacts: oAuth
storage = Storage('/home/almo/dev/keys/ex1/oAuth_credentials.dat')
credentials = storage.get()
if credentials is None or credentials.invalid:
PEOPLE_API='https://www.googleapis.com/auth/contacts.readonly'
flow = flow_from_clientsecrets('/home/almo/dev/keys/ex1/oAuth_key.json',
scope=[PEOPLE_API])
credentials = run_flow(flow, storage)
http = credentials.authorize(httplib2.Http())
service = build('people','v1',http=http)
request = service.people().connections().list(resourceName='people/me',
pageSize=500)

25. analyze_image
api_key = json.load(open('/home/almo/dev/keys/ex1/api_key.json'))['api_key']
service = discovery.build('vision','v1',developerKey=api_key)
service_request = service.images().annotate(body={
'requests': [{
'image': {
'content': image_content.decode('UTF-8')
},
'features': [{
'type': 'LABEL_DETECTION',
'maxResults': 3
}]
}]
}

26. Data

27. face; 0,92830354; https://lh3.googleusercontent.com/-c3M1gn6ougg/AAAAAAAAAAI/AAAAAAAAAds/cTIrpGhktfw/photo.jpg?sz=250
text; 0,93046468; https://lh4.googleusercontent.com/-GFVyrVlgMy4/AAAAAAAAAAI/AAAAAAAAABE/u3xVd9eJgf8/photo.jpg?sz=250
font; 0,85384184; https://lh4.googleusercontent.com/-GFVyrVlgMy4/AAAAAAAAAAI/AAAAAAAAABE/u3xVd9eJgf8/photo.jpg?sz=250
line; 0,70535356; https://lh4.googleusercontent.com/-GFVyrVlgMy4/AAAAAAAAAAI/AAAAAAAAABE/u3xVd9eJgf8/photo.jpg?sz=250
eyebrow; 0,98022038; https://lh5.googleusercontent.com/-5c9gdP9nX9M/AAAAAAAAAAI/AAAAAAAAGt4/FoZEEVA8F68/photo.jpg?sz=250
hair; 0,96653992; https://lh5.googleusercontent.com/-5c9gdP9nX9M/AAAAAAAAAAI/AAAAAAAAGt4/FoZEEVA8F68/photo.jpg?sz=250
face; 0,95101357; https://lh5.googleusercontent.com/-5c9gdP9nX9M/AAAAAAAAAAI/AAAAAAAAGt4/FoZEEVA8F68/photo.jpg?sz=250
person; 0,92170084; https://lh4.googleusercontent.com/-yVWpXcqQfXU/AAAAAAAAAAI/AAAAAAAAB5w/rqxRrJHgk_0/photo.jpg?sz=250
news; 0,63342041; https://lh4.googleusercontent.com/-yVWpXcqQfXU/AAAAAAAAAAI/AAAAAAAAB5w/rqxRrJHgk_0/photo.jpg?sz=250
professional; 0,61274487;
https://lh4.googleusercontent.com/-yVWpXcqQfXU/AAAAAAAAAAI/AAAAAAAAB5w/rqxRrJHgk_0/photo.jpg?sz=250
drawer; 0,80023241; https://lh6.googleusercontent.com/-Qf9SSsIUktA/AAAAAAAAAAI/AAAAAAAAABg/u6zPUNXCYFs/photo.jpg?sz=250
furniture; 0,79278195; https://lh6.googleusercontent.com/-Qf9SSsIUktA/AAAAAAAAAAI/AAAAAAAAABg/u6zPUNXCYFs/photo.jpg?sz=250
product; 0,76023591; https://lh6.googleusercontent.com/-Qf9SSsIUktA/AAAAAAAAAAI/AAAAAAAAABg/u6zPUNXCYFs/photo.jpg?sz=250
eyewear; 0,97702742; https://lh3.googleusercontent.com/-ihQNk3ewmzQ/AAAAAAAAAAI/AAAAAAAAAMk/EEEylEyriNE/photo.jpg?sz=250
hair; 0,96766639; https://lh3.googleusercontent.com/-ihQNk3ewmzQ/AAAAAAAAAAI/AAAAAAAAAMk/EEEylEyriNE/photo.jpg?sz=250
sunglasses; 0,96445274; https://lh3.googleusercontent.com/-ihQNk3ewmzQ/AAAAAAAAAAI/AAAAAAAAAMk/EEEylEyriNE/photo.jpg?sz=250
person; 0,92747426; https://lh4.googleusercontent.com/--_BxhkQPYfA/AAAAAAAAAAI/AAAAAAAAACA/1pN6-Chy8EI/photo.jpg?sz=250
person; 0,96007371; https://lh3.googleusercontent.com/-sX8l_lv_-7w/AAAAAAAAAAI/AAAAAAAAAPU/ApQpBMPbcdc/photo.jpg?sz=250
face; 0,95332307; https://lh3.googleusercontent.com/-sX8l_lv_-7w/AAAAAAAAAAI/AAAAAAAAAPU/ApQpBMPbcdc/photo.jpg?sz=250
Raw Data

28. 160 different labels
Max Freq.: 200
Min Freq. : 1

29. person 200 0,9320951099
hair 140 0,9609928544
face 139 0,9489352931
font 136 0,7606724908
text 130 0,925080287
blue 114 0,9112923658
facial hair 36 0,8802876539
nose 34 0,8859786603
profession 30 0,569073382
hairstyle 25 0,7532089968

30. cartoon 23 0,8588066957
professional 23 0,6149233535
glasses 20 0,8234816515
facial expression 14 0,9502550086
eyebrow 12 0,9559630675
black and white 11 0,9199305709
eyewear 11 0,9767648145
logo 11 0,7749610755
head 9 0,7432496333
clothing 7 0,9151270214

32. [{}]

34. Planning our next
Party

35. "hair", 0.9559916, "person", 0.94347906, "face", 0.92830354
"text", 0.9304647, "font", 0.85384184, "line", 0.70535356
"eyebrow", 0.9802204, "hair", 0.9665399, "face", 0.9510135
"person", 0.92170084, "news", 0.63342035, "professional", 0.61274487
"drawer", 0.8002325, "furniture", 0.792782, "product", 0.760235
"eyewear", 0.9770274, "hair", 0.9676664, "sunglasses", 0.96445274
"person", 0.92747426,
"https://lh4.googleusercontent.com/--_BxhkQPYfA/AAAAAAAAAAI/AAAAAAAAACA/1pN6-Chy8EI/photo.jpg
?sz=250"
"green", 0.9307698, "text", 0.92834556, "font", 0.8631033
"hair", 0.98155975, "face", 0.95545304, "eyebrow", 0.93590355
"face", 0.9523797, "person", 0.94760686, "hair", 0.94507515
"hair", 0.9731342, "face", 0.94925183, "person", 0.9371813
"hair", 0.94741917, "person", 0.9436425, "hairstyle", 0.7414854
"person", 0.925232, "people", 0.9086431, "male", 0.83032143
"person", 0.95530343, "face", 0.94757956, "nose", 0.86752254
"face", 0.96074444, "hair", 0.9606222, "eyebrow", 0.9451414
"face", 0.9664352, "hair", 0.9561741, "nose", 0.9222636
"phenomenon", 0.94444287, "celestial event", 0.53744316, "aurora", 0.52995497
"face", 0.9625666, "hair", 0.9514838, "facial expression", 0.94977105
"product", 0.80306137, "font", 0.77923214, "logo", 0.69078964
"black and white", 0.9267871, "person", 0.8998944, "photography", 0.8296365

36. Training
mode!

37. “invited”, "hair", 0.9559916, "person", 0.94347906, "face", 0.92830354
“excluded”, "text", 0.9304647, "font", 0.85384184, "line", 0.70535356
“excluded”, "eyebrow", 0.9802204, "hair", 0.9665399, "face", 0.9510135
“invited”, "person", 0.92170084, "news", 0.63342035, "professional",
0.61274487
“excluded”, "drawer", 0.8002325, "furniture", 0.792782, "product", 0.760235
“excluded”, "eyewear", 0.9770274, "hair", 0.9676664, "sunglasses", 0.96445274
“excluded”, "green", 0.9307698, "text", 0.92834556, "font", 0.8631033
“excluded”, "hair", 0.98155975, "face", 0.95545304, "eyebrow", 0.93590355
“invited”, "face", 0.9523797, "person", 0.94760686, "hair", 0.94507515
“invited”, "hair", 0.9731342, "face", 0.94925183, "person", 0.9371813
“invited”, "hair", 0.94741917, "person", 0.9436425, "hairstyle", 0.7414854
“invited”, "person", 0.925232, "people", 0.9086431, "male", 0.83032143
“invited”, "person", 0.95530343, "face", 0.94757956, "nose", 0.86752254
“excluded”, "face", 0.96074444, "hair", 0.9606222, "eyebrow", 0.9451414
“excluded”, "face", 0.9664352, "hair", 0.9561741, "nose", 0.9222636

38. Prediction
Mode!

39. "hair", 0.9559916, "person", 0.94347906, "face", 0.92830354 “invited”
"text", 0.9304647, "font", 0.85384184, "line", 0.70535356 “excluded”
"eyebrow", 0.9802204, "hair", 0.9665399, "face", 0.9510135 “excluded”
"person", 0.92170084, "news", 0.63342035, "professional", 0.61274487 “invited”
"drawer", 0.8002325, "furniture", 0.792782, "product", 0.760235 “excluded”
"eyewear", 0.9770274, "hair", 0.9676664, "sunglasses", 0.96445274 “excluded”
"green", 0.9307698, "text", 0.92834556, "font", 0.8631033 “excluded”
"hair", 0.98155975, "face", 0.95545304, "eyebrow", 0.93590355 “excluded”
"face", 0.9523797, "person", 0.94760686, "hair", 0.94507515 “invited”
"hair", 0.9731342, "face", 0.94925183, "person", 0.9371813 “invited”
"hair", 0.94741917, "person", 0.9436425, "hairstyle", 0.741485 4 “invited”
"person", 0.925232, "people", 0.9086431, "male", 0.83032143 “invited”
"person", 0.95530343, "face", 0.94757956, "nose", 0.86752254 “invited”
"face", 0.96074444, "hair", 0.9606222, "eyebrow", 0.9451414 “excluded”
"face", 0.9664352, "hair", 0.9561741, "nose", 0.9222636 “excluded”

42. Curating Online Content with
Google ML APIs
Andres L. Martinez a.k.a almo
Google Developer Relations Manager
@davilagrau