Research 
Large-Scale 
Evaluation of Call- 
Availability 
Prediction 
Martin Pielot, 
Telefónica Research 
ACM UbiComp’14,...
Phone calls reach us 
anywhere, 
anytime 
makelessnoise. Phon-ey Call. via Flickr, Jul 07, 2006 (CC BY 2.0)
30% 
of all calls 
are missed 
WilliamTheaker. Hamhung cyclist. via Wikipedia, Apr 17, 2012 (CC
Please raise hands: 
Who of you would take a call 
right now? 
Kaz. Hände. Via Pixabay, Nov 26, 2013. (Public Domain
Interruption!
Callers want to know: 
 Location and time, 
 physical, social, 
emotional availability, 
 and current activity. 
De Guz...
Callers want to know: 
 Location and time, 
 physical, social, 
emotional availability, 
 and current activity. 
De Guz...
People have concerns sharing too much 
contextual information Knittel et al. 2011 
"No trespassing" by Djuradj Vujcic - Ow...
Availability 
Location 
Time 
Presence of others 
Current Activity 
… 
Machine-Learning 
User Model 
…
Related Approaches 
Horvitz et al. 
2005 
Using calendar details from Outlook to 
predict cost of interruption by call
Related Approaches 
Horvitz et al. 
2005 
Using calendar details from Outlook to 
predict cost of interruption by call 
Ro...
Related Approaches 
Horvitz et al. 
2005 
Using calendar details from Outlook to 
predict cost of interruption by call 
Ro...
Opportunities to Advance Line of Work 
(1)Actual reaction 
(1) Sample selection & size
Study
On shake
Anonymous 
logs
Sep 17, 2014, 15:20 
Screen 
Status 
Reaction to 
the call 
Proximity 
Sensor 
Day and 
Time 
Ringer 
mode 
Charging
Results
31,311 calls 
from 418 distinct user
Extracted 15 Basic Features 
Category Feature 
Last Active Last ringer change (time) 
Last Active Last screen change (time...
Prediction 
Random Forest (10 trees) 
Classes: available | not available 
Accuracy 83.2% (κ=.646) 
(10-fold cross-validati...
19,175 calls picked up 
61.2% baseline 
accuracy
Advantages of large data set
Model accuracy over time 
85 
83 
81 
79 
77 
75 
73 
71 
Accuracy (%) 
Number of instances (phone calls)
Random Forest Model 
Personalized model 
Subset of 120 calls: 87.0% (κ=.640)
Features Ranked by Prediction Power 
Category Feature Mean Rank 
Last Active Last ringer change (time) 1 
Last Active Last...
Application
Mute the ringer
Communicate 
Non-Availability
Communicate 
Non-Availability Likely to be 
unavailable
Large-Scale Evaluation of Call-Availalability Prediction. 
First large-scale study (31,311 calls) of call-availability 
pr...
Large-Scale Evaluation of Call-Availability Prediction
Nächste SlideShare
Wird geladen in …5
×

Large-Scale Evaluation of Call-Availability Prediction

1.269 Aufrufe

Veröffentlicht am

Slides on Large-Scale Evaluation of Call-Availability Prediction - talk given at UbiComp '14 conference in Seattle.

Veröffentlicht in: Wissenschaft
0 Kommentare
0 Gefällt mir
Statistik
Notizen
  • Als Erste(r) kommentieren

  • Gehören Sie zu den Ersten, denen das gefällt!

Keine Downloads
Aufrufe
Aufrufe insgesamt
1.269
Auf SlideShare
0
Aus Einbettungen
0
Anzahl an Einbettungen
717
Aktionen
Geteilt
0
Downloads
5
Kommentare
0
Gefällt mir
0
Einbettungen 0
Keine Einbettungen

Keine Notizen für die Folie
  • http://commons.wikimedia.org/wiki/File:2007Computex_e21Forum-MartinCooper.jpg#mediaviewer/File:2007Computex_e21Forum-MartinCooper.jpg

  • makelessnoise. Phon-ey Call. via Flickr, Jul 07, 2006 (CC BY 2.0)
    https://www.flickr.com/photos/makelessnoise/195088755/
  • http://en.wikipedia.org/wiki/Economy_of_North_Korea#mediaviewer/File:Hamhung_cyclist.jpg
  • Kaz. Hände. Via Pixabay, Nov 26, 2013. (Public Domain CC0)
    http://pixabay.com/p-220163/
  • http://commons.wikimedia.org/wiki/File:White_Diamonds_Party.jpg#mediaviewer/File:White_Diamonds_Party.jpg
  • http://commons.wikimedia.org/wiki/File:White_Diamonds_Party.jpg#mediaviewer/File:White_Diamonds_Party.jpg
  • People are only willing to share some contextual information, such as current location, current activity, or presence of appointments.
    [Knittel et al. 2011]
    http://en.wikipedia.org/wiki/Trespass#mediaviewer/File:No_trespassing_by_Djuradj_Vujcic.jpg
    "No trespassing by Djuradj Vujcic" by Djuradj Vujcic - Own work. Licensed under Creative Commons Attribution-Share Alike 3.0 via Wikimedia Commons - http://commons.wikimedia.org/wiki/File:No_trespassing_by_Djuradj_Vujcic.jpg#mediaviewer/File:No_trespassing_by_Djuradj_Vujcic.jpg
  • Be present in the lives of a large number of users
  • 3,320, that is 10.8% of the calls where muted by shaking the phone and not answering the call
    That is, they were most likely interruptive
  • http://pixabay.com/de/geschlossen-angeln-gegangen-315859/
  • Large-Scale Evaluation of Call-Availability Prediction

    1. 1. Research Large-Scale Evaluation of Call- Availability Prediction Martin Pielot, Telefónica Research ACM UbiComp’14, Sep, 2014, Seattle, USA Wed, Sep 17, 2014 – 14:00 – 15:30 Session: Interruptability & Notifications "2007Computex e21Forum-MartinCooper" by Rico Shen. Via Wikimedia. CC-SA 3.0.
    2. 2. Phone calls reach us anywhere, anytime makelessnoise. Phon-ey Call. via Flickr, Jul 07, 2006 (CC BY 2.0)
    3. 3. 30% of all calls are missed WilliamTheaker. Hamhung cyclist. via Wikipedia, Apr 17, 2012 (CC
    4. 4. Please raise hands: Who of you would take a call right now? Kaz. Hände. Via Pixabay, Nov 26, 2013. (Public Domain
    5. 5. Interruption!
    6. 6. Callers want to know:  Location and time,  physical, social, emotional availability,  and current activity. De Guzman et al. 2007 "White Diamonds Party" by Club Skirts Dinah Shore Weekend - Own work. via Wikimedia Commons. CC BY-SA 3.0 -
    7. 7. Callers want to know:  Location and time,  physical, social, emotional availability,  and current activity. De Guzman et al. 2007 "White Diamonds Party" by Club Skirts Dinah Shore Weekend - Own work. via Wikimedia Commons. CC BY-SA 3.0 - Callees react depend. on:  Location and time,  Presence of others,  and current activity. Danninger et al. 2006
    8. 8. People have concerns sharing too much contextual information Knittel et al. 2011 "No trespassing" by Djuradj Vujcic - Own work. via Wikimedia Commons. CC BY-SA 3.0.
    9. 9. Availability Location Time Presence of others Current Activity … Machine-Learning User Model …
    10. 10. Related Approaches Horvitz et al. 2005 Using calendar details from Outlook to predict cost of interruption by call
    11. 11. Related Approaches Horvitz et al. 2005 Using calendar details from Outlook to predict cost of interruption by call Rosenthal et al. 2011 Use ESM to train phones to mute ringer in certain situations
    12. 12. Related Approaches Horvitz et al. 2005 Using calendar details from Outlook to predict cost of interruption by call Rosenthal et al. 2011 Use ESM to train phones to mute ringer in certain situations Pejovic and Musolesi 2014 Identifying opportune moments for mobile device-based interruptions
    13. 13. Opportunities to Advance Line of Work (1)Actual reaction (1) Sample selection & size
    14. 14. Study
    15. 15. On shake
    16. 16. Anonymous logs
    17. 17. Sep 17, 2014, 15:20 Screen Status Reaction to the call Proximity Sensor Day and Time Ringer mode Charging
    18. 18. Results
    19. 19. 31,311 calls from 418 distinct user
    20. 20. Extracted 15 Basic Features Category Feature Last Active Last ringer change (time) Last Active Last screen change (time) Last Active Last (un)plugged (time) Last Active Last call (time) Currently Active Screen status Currently Active Pitch of phone Relationship How often called by caller Context Day of the week Context Hour of the day Context Charger (un)plugged Context Ringer mode Context Last call silenced Context Activity / Acceleration Context Screen (not) covered Context Last call picked
    21. 21. Prediction Random Forest (10 trees) Classes: available | not available Accuracy 83.2% (κ=.646) (10-fold cross-validation)
    22. 22. 19,175 calls picked up 61.2% baseline accuracy
    23. 23. Advantages of large data set
    24. 24. Model accuracy over time 85 83 81 79 77 75 73 71 Accuracy (%) Number of instances (phone calls)
    25. 25. Random Forest Model Personalized model Subset of 120 calls: 87.0% (κ=.640)
    26. 26. Features Ranked by Prediction Power Category Feature Mean Rank Last Active Last ringer change (time) 1 Last Active Last screen change (time) 2 Currently Active Screen status 3.6 Last Active Last (un)plugged (time) 5.4 Last Active Last call (time) 6.8 Context Activity / Acceleration 7.3 Relationship How often called by caller 7.6 Context Day of the week 9.4 Context Hour of the day 10 Context Charger (un)plugged 10.1 Context Ringer mode 11.4 Context Last call silenced 12.4 Currently Active Pitch of phone 12.5 Context Screen (not) covered 13 Context Last call picked 14.1
    27. 27. Application
    28. 28. Mute the ringer
    29. 29. Communicate Non-Availability
    30. 30. Communicate Non-Availability Likely to be unavailable
    31. 31. Large-Scale Evaluation of Call-Availalability Prediction. First large-scale study (31,311 calls) of call-availability prediction Prediction possible with 15 basic features 83% accuracy (generic models) 87% accuracy (personalized models) Strongest 5 predictors 4 features regarding time of last activity Screen status Use cases Mute ringer on unavailability Allow caller to check availability Large-Scale Evaluation of Call- Availability Prediction Martin Pielot, Telefónica Research martin.pielot@telefonica.com ACM UbiComp’14, Sep, 2014, Seattle, USA Wed, Sep 17, 2014 – 14:00 – 15:30 Interruptability & Notifications Q&A

    ×