From Tizen Developer Conference 2017- Through Context APIs, apps can detect the user's activities in the physical world. Along with the context awareness, Machine Learning and AI algorithms can make the user's app experience much more personalised. Opportunities, tools and lessons of these apps will be discussed. The talk will focus on User Experience, development and future applications. Artificial Intelligence and Machine Learning models on top of the sensor, context & user data will allow apps to get a better understanding about the user.

1. 1Tizen apps with
Context Awareness,
powered by AI
by Shashwat Pradhan, CEO Emberify

2. 2
Context refers to information that characterizes a
situation, between:
• Apps
• People
• Surrounding environment
• Making apps smarter and more relevant to every
individual user by understanding them
Introduction

3. 3
Contextual apps understand what’s going on in the
user’s digital and physical world
Introduction

4. 4
• Today, an average smartphone has about 10
sensors
• Contextual data: Current location, time, surrounding
brightness, user activity
• User’s digital world: Apps being used, Facebook API,
Instagram API
• Wearables & IoTs are bringing in many new data
points
Introduction

5. 5
• Alarm based on weather & traffic to work by location
sensing
• Phone goes on vibrate based on proximity to office
or a movie theatre
• Reminders based on travel tickets on email
• Adaptive UI – App theme changes according to
surrounding brightness
Some contextual experiences

6. 6
• Sense, understand and adapt
• Get user data from sensors or social networks
• Build algorithms to understand the contextual
data
• Personalize content & provide proactive
recommendations
Contextual Lifecycle
Sense
Understand
Adapt

7. 7
Example of a context aware alarm app
• Senses the location of the user
• Understands the current weather & traffic
through an API and usual waking up time
• Adapts by letting the user sleep longer or
shorter based on these conditions
Contextual Lifecycle
Sense
Understand
Adapt

8. 8
Tizen 2.4.0 on mobile came with a large set of Context
APIs so developers don’t need to access sensors
directly
• Activity Recognition (Wearable Also)
• Contextual History
• Contextual Trigger
• Gesture Recognition (Wearable Also)
Context with Tizen

9. 9
• Activity Recognition – Stationary, walking, travelling,
running with accuracy
• Contextual History – Device Usage Patterns like app
usage, peak time and commonly used settings
• Contextual Trigger – Based on a contextual event
the app or notification is triggered
• Gesture Recognition – Shake, Tilt, Snap, Orientation
and other gestures are detected
Context with Tizen

10. 10
• Apart from the data from the Context API’s, user
data can be obtained from Social API’s
• Calendar and contacts can provide apps with
information about the user’s physical world
• Using Sensor API’s sensors can be directly polled
• For most cases Tizen Context API’s are doing the
sense and understand part for developers
Context with Tizen

11. 11
• With Artificial Intelligence, context aware apps can
be more proactive and intelligent
• AI algorithms can help make future context
predictions
• Context awareness can make AI applications like
Chatbots smarter
Context with AI

12. 12
• Example- The chatbot needs to recommend the user
places when it’s raining
• Chatbot: Since it is/is going to be raining outside I
am recommending you indoor places
• Gives the Chatbot more human intelligence
properties considering the user’s context
Context with AI

13. 13
AI can help with contextual apps with:
• Sensing with abstraction of data
• Sensors can generate huge amounts of data from
which AI algorithms can help extract the relevant
data
• Also, understanding data through audio, images and
video can be done through AI branches like
computer vision & speech recognition
Context with AI

14. 14
• User Profiling
• Gathering preferences of a user through sensors,
behavior, social networks or even explicitly
• Personalizing the content and Adaptive nature of the
app according to different user profiles
• AI techniques like association rules or case-based
reasoning can be used
Context with AI

15. 15
• User Profiling
• Example- News app profiles users to give relevant
articles based on user’s interests
• General preferences (Time), Short-term interests
(Sporting event) & Long-term interests (Politics)
• Keywords can be weighted to prioritize stories
Context with AI

16. 16
• Context Reasoning
• Based on the context situation an app may need to
adapt
• Examining the contextual information and making a
decision based on rules and logic
• The decision logic can further evolve with Machine
Learning algorithms
Context with AI

17. 17
• ML algorithms learn from and make predictions on
data
• ML algorithms work on models have to be made
based on sample inputs
• Enables context prediction – which sensor data
could be most important in the future
Context with Machine Learning

18. 18
• Using a combination of sensors, Machine Learning
models can be used to determine user activity
• Extract sensor data and train ML models
• Multiple context data used together can give more
specific information about the user
• Example Accelerometer & Barometer can be
used together to detect walking vs cycling
Context with Machine Learning
Sensor Data
Machine Learning
Server / On-device
model

19. 19
• ML algorithms make sense of noisy/conflicting data
from sensors
• Large datasets are useful to train & fine tune
Machine Learning models
• ML algorithms use raw sensor data to churn out
signals based on training models like high level
activities
Context with Machine Learning

20. 20
Six technology forces powering contextual
apps:
• Mobile
• Social Media
• Sensor evolution
• Cloud & Big Data
• Wearable & other IoTs
• Artificial Intelligence
Technology Powering Contextual Apps

21. 21
• Launchify – App recommendation widget
• Predicts which app the user needs right now in the
widget
• Context signals measured for prioritizing information:
• User travelling
• App Usage Patterns
• Location
Case Study

22. 22
• Launchify – App recommendation widget
• Using Machine Learning algorithms to learn based
on user behavior
• Simple weighing algorithm to give each contextual
parameter weight to priorities
• Senses where, how long, how often, what situations
are the apps being used
Case Study

23. 23
• Some common sense assumptions are needed in
addition to the sensor data based on general human
behavior to get more accuracy
• Sometimes sensors can give us conflicting data.
• Use multiple sensors to confirm it
• Simple logic can be applied to the algorithm like
repeating of a certain event occurrence before
counting it to avoid random events
Experiences with Contextual Apps

24. 24
• Proactive Recommendations
• Recommending the user outdoor places on their lockscreen if
there's a chance of rain
• Lifelogging
• Quantified Self apps to track the user’s life automatically
• Adaptive User Experience
• Automatically changing the theme according to sorrounding
brightness
Use Cases

25. 25
• Context is the secret sauce making an app smart &
unique
• Foursquare doubled down on locations services to
give proactive recommendations of food when you’re
sitting at a restaurant
• The contextual fabric can provide a personalized
experience
• New value for users can help apps find interest in
App Stores crowded with millions of apps
Use Cases

26. 26
• Contextual data is not always accurate
• Allow the user to correct and edit the contextual data
• Eg. Slow driving is often confused as cycling
• Machine Learning models take huge amounts of
data to train for accuracy
Limitations

27. 27
• With Context Aware apps you need to be transparent
what the app is doing with the user’s data
• There needs to be a clear privacy policy
• User’s should be able to disable the services
• Encryption and security protocols need to be in
place
Limitations - Privacy

28. 28
• Rather than providing the wow factor some
contextual apps go over the freaky line
• Nokia’s Trapster (Similar to Waze) would allow it’s
users to stalk other users accurately
• Huge user privacy & trust issues
Limitations - Privacy

29. 29
• Sensors and background services can consume lots
of battery life
• Data should be polled on triggers rather than a timer
• Rather than going to the sensor every time it would
be more efficient to get data through an app that just
polled the data
Limitations – Battery life

30. 30
• More IoTs and wearables will bring in new sets of
data and better quality too
• Smart Cars & Smart Homes will also add to user
information
• Apps will be more automatic with better sensing
• Contextual Apps will be more proactive in nature
• Smartphone OS’s will take more contextual
information to become more intelligent
Future

31. 31
• Apps will be ‘Headless’, will require minimum
interface interaction
• Smart Notifications, voice and chatbots will be the
new interfaces since apps will need lesser input with
contextual information
Future

32. 32
Thank you
shashwat@emberify.com
@shashwatpradhan
http://emberify.com
Questions