How to do MobileHCI Research in the Large

1.836 Aufrufe

Veröffentlicht am

Since the introduction of application stores for mobile devices there has been an increasing interest to use this distribution platform to collect user feedback. Mobile application stores can make research prototypes widely available and enable to conduct user studies "in the wild" with participants from all over the world. Using apps as an apparatus goes beyond just distributing research prototypes. Consider apps as a tool for research means distributing specifically designed prototypes in order to extend our understanding of mobile HCI. In this tutorial we will provide an overview about recent research in this domain. It will be shown that stringent tasks and users´ motivation are crucial aspects. We will discuss how to design app-based experiments, what kind of users one can expect, and how to avoid ethical and legal issues.

Veröffentlicht in: Bildung
0 Kommentare
0 Gefällt mir
  • Als Erste(r) kommentieren

  • Gehören Sie zu den Ersten, denen das gefällt!

Keine Downloads
Aufrufe insgesamt
Auf SlideShare
Aus Einbettungen
Anzahl an Einbettungen
Gefällt mir
Einbettungen 0
Keine Einbettungen

Keine Notizen für die Folie

How to do MobileHCI Research in the Large

  1. 1. How  to  do  Mobile  HCI  Research  in  the  large?    Niels  Henze    University  of  Oldenburg  Media  Informa@cs  and    Mul@media  Systems    OFFIS  -­‐  Ins@tute  for  Informa@on  Technology  Intelligent  User  Interface  Group    
  2. 2. …  but  lets  start  with  a  ques@on:    Who  of  you  ever  par@cipated  in  a  user  study?  
  3. 3. do  you  think  that  any  of  these  guys   ever  did?  
  4. 4. Outline  1.  Limita@ons  of  common  studies  2.  Into  the  large  3.  Types  of  studies  4.  What  is  so  special?  5.  What  works  for  me  6.  Wrap  up    
  5. 5.    User  studies  at   MobileHCI  2010    20%  acceptance  rate    43  short+long  papers    
  6. 6.     User  studies  at   MobileHCI  2010    20%  acceptance  rate    43  short+long  papers    subjects  per  paper  hXp://  
  7. 7.     User  studies  at   MobileHCI  2010    20%  acceptance  rate    43  short+long  papers    subjects  per  paper    subject’s  gender    hXp://  
  8. 8. all  with  a  university  degree,  recruited  in     the  Ins@tute  community     students  or  employees  at  our   university   User  studies  at   recruited  through  flyers,  posters  and   various  mailing  lists  at  the  university   MobileHCI  2010    20%  acceptance  rate   10  university  students  and  2  par@cipants    43  short+long  papers   are  marke@ng  professionals    subjects  per  paper   undergraduate  or  graduate  students  at    subject’s  gender   the  local  university  studying  a  variety  of   majors    o]en  a  biased  sample   university  students    most  subjects  were  students  with  a  background  in  computer  sciences   most  par@cipants  were  students  studying  or  working  in  the  University  of  Glasgow   members  in  a  joint  research  project  hXp://  
  9. 9. Some  male  students  from  the  lab  took  part  in  our  study...  Small  sample  size  isn’t  necessarily  an  issue  for  a   study  Not  every  study  needs  a  perfect  sample  of  the   popula@on    Focussing  on  studies  with  few  subjects  prevents   finding  subtle  differences  We  stew  in  our  own  juices  if  using  our  own   students  by  default  
  10. 10. Some  mo@va@on  Large  numbers  are  expensive  in  the  lab   –  1,000  subjects  for  an  hour  -­‐>  10,000€   –  1,000  subjects  for  an  hour  -­‐>  6  month   –  1,000  subjects  from  around  the  world  -­‐>  impossible    Different  contexts  are  hard  to  address   –  We  have  no  subway  in  Oldenburg   –  Don’t  want  to  pay  the  flight  for  my  par@cipant     –  And  what  are  the  relevant  contexts  anyway?  
  11. 11.    Example  of  gehng  large…   Target  selec@on  on   mobile  phones    thirty  right-­‐handed  subjects    different  target  loca@ons   and  sizes  [Park2008MobileHCI]  
  12. 12.     Target  selec@on  on   mobile  phones    thirty  right-­‐handed  subjects    different  target  loca@ons   and  sizes     Taps  are  skewed    fixed  posture    single  device    Korean  students      vague  results  [Park2008MobileHCI]  
  13. 13.    …same  thing  in  the   large    game  published  on  the   Android  Market      
  14. 14.    …same  thing  in  the   large    game  published  on  the   Android  Market    we  inform  the  player   about  the  study    
  15. 15.    …same  thing  in  the   large    game  published  on  the   Android  Market    we  inform  the  player   about  the  study    just  looks  like  an  ordinary   game      
  16. 16.    …same  thing  in  the   large    game  published  on  the   Android  Market    we  inform  the  player   about  the  study    just  looks  like  an  ordinary   game    par@cipants  get  some   introduc@on      
  17. 17.    …same  thing  in  the   large    game  published  on  the   Android  Market    we  inform  the  player   about  the  study    just  looks  like  an  ordinary   game    par@cipants  get  some   introduc@on      
  18. 18.    …same  thing  in  the   large    game  published  on  the   Android  Market    we  inform  the  player   about  the  study    just  looks  like  an  ordinary   game    par@cipants  get  some   introduc@on    they  tap  the  targets      
  19. 19.    …same  thing  in  the   large    game  published  on  the   Android  Market    we  inform  the  player   about  the  study    just  looks  like  an  ordinary   game    par@cipants  get  some   introduc@on    they  tap  the  targets    we  vary  targets’  size  and   posi@on      
  20. 20.    …same  thing  in  the   large    game  published  on  the   Android  Market    we  inform  the  player   about  the  study    just  looks  like  an  ordinary   game    par@cipants  get  some   introduc@on    they  tap  the  targets    we  vary  targets’  size  and   posi@on    there  is  even  a  high  score   list  
  21. 21.      published  on  the   Android  Market    100,000  installa@ons  in   three  months    120  million  touch  events    more  than  hundred   different  devices    players  from  all  over  the   world  
  22. 22. [Park2008MobileHCI]  
  23. 23. [Henze2011MobileHCI]  
  24. 24. Types  of  work  Proof  of  concept   –  Showing  that  an  idea/concept/product  works   –  Lots  of  users,  good  ra@ngs,  posi@ve  comments,  ...  App  stores  as  research  tool   –  Experience  report   –  Ethical  and  legal  issues    Inves@ga@ng  app-­‐specific  aspects   –  How  a  specific  app  is  used   –  Compare  different  visualiza@ons  Observing  general  aspects   –  Learn  about  how  people  and  devices  behave   –  How  are  apps  how,  how  people  touch  the  screen,  ...  
  25. 25. Proof  of  concept  
  26. 26.       Smule’s  iPhone   Ocarina    music  instrument  for  the   iPhone    million  installa@ons  [Wang2009NIME]  
  27. 27.       Shapewriter    developed  gesture-­‐based   keyboard  +  notepad    qualita@ve  feedback  from   App  Store  comments  [Zhai2009CHI]  
  28. 28. App  stores  as  research  tool  
  29. 29.       Into  the  wild  with   Hungry  Yoshi    loca@on  based  game  for   the  iPhone      94,642  unique  downloader    inves@gated  how  to  get   subjec@ve  feedback  [McMillan2010Pervasive]  
  30. 30.  100%     83,68%   81,31%   80%     Experience  from           60%   54,76%   5  Studies   40%    compare  amount  of   collected  data   20%    experience  with  collec@ng   7,32%   qualita@ve  data   0,46%    discuss  internal  and   0%   external  validity   [Henze2011IJMHCI]  
  31. 31. Inves@ga@ng  app-­‐specific  aspects  
  32. 32.       Ra@ngs  for  Mobile   Applica@ons    compare  amount  of   collected  data    experience  with  collec@ng   qualita@ve  data    discuss  internal  and   external  validity  [Girardello2010DSZ]  
  33. 33.       Compare  off-­‐screen   visualisa@ons    using  repeated  measures    using  a  tutorial  for  a  map   applica@on  [Henze2010MobileHCI]   [Henze2010MobileHCI]  
  34. 34.       Compare  off-­‐screen   visualisa@ons    using  repeated  measures    using  a  tutorial  for  a  map   applica@on    and  using  a  simple  game   (which  worked  much   beXer!)  [Henze2010MobileHCI]   [Henze2010MobileHCI]  
  35. 35. Observing  general  aspects  
  36. 36. Falling  Asleep  with  …  appazaar   [Böhmer2011MobileHCI]  
  37. 37. A  Study  of  BaXery  Life   [Ferreira2011Pervasive]  
  38. 38. app  stores  as    a   inves@ga@ng  app-­‐ inves@ga@ng  proof  of  concept   research  tool   specific  aspects   general  aspects   [Wang2009NIME]   [McMillan2010RiL]   [Girardello2010DSZ]   [Hood2011IJTR]   [McMillan2010Pervasive]   [Zhai2009CHI]   [Riccamboni2010IB]   [Henze2011MobileHCIa]   [Henze2011IJMHCI]  [Gilbertson2008CiE]   [Kuhn2010MM]   [Henze2011MobileHCIb]   [Miluzzo2010RiL]   [Watzdorf2010LocWeb]   [Poppinga2010OMUE]   [Yan2011MobiSys]   [Ferreira2011Pervasive]   [Oliver2010HotPlanet]   [Budde2010IoT]   [Morrison2010RiL]   [Buddharaju2010CHI]   [Karpischek2011RiL]   [Sahami2011CHI]   [Henze2010MobileHCI]   [Verkasalo2010MB]   [Pielot2011ELV]   [Henze2010NordiCHI]   [Böhmer2011MobileHCI]   [Cramer2010UbiComp]   [Morrison2011CHI]   [Henderson2009HotPlanet]   [Norcie2011ELV]   Ethics  and  legal  issues  
  39. 39. but  what  is  special  about  app  store  studies?  
  40. 40. App-­‐based  vs.  other  studies  Common  con-­‐ Mining  exis@ng   App-­‐based  trolled  studies   data   studies  Few  par@cipants   Many  par@cipants   Many  par@cipants  Ar@ficial  context   Natural  context   Natural  context   Defined  tasks                    Defined  task   No  tasks   (if  needed)  Total  control  over   Weak  control  over   No  control  par@cipants   par@cipants  Heavily  biased   Biased  to  unbiased   Unbiased  sample  sample   sample  
  41. 41. You  have  to  “sell”  your  study  The  study  has  a  goal   –  Collect  informa@on  about  specific  behaviour   –  Performance  for  a  specific  task  Users  have  to  install  the  app  on  their  own  will   –  App  needs  a  purpose   –  Good  ra@ngs,  high  ranking  Find  a  compromise   –  Maintain  the  goals  of  the  study   –  AXract  sufficient  par@cipants  
  42. 42. Types  of  apps   Applica@ons   Games   Widgets  
  43. 43.    100.000   90.000     80.000   Par@cipants   70.000    How  do  we  count  the   60.000   number  of  par@cipant?   50.000   40.000   30.000   20.000   10.000   0   installa@ons   opt-­‐in   ac@ve  users   [McMillan2010Pervasive]   [Morrison2010RiL]  
  44. 44.     US  Android  users   US  popula@on    60%   Par@cipants  50%    How  do  we  count  the   number  of  par@cipant?  40%    A  good  sample  of  the  30%   popula@on?  20%  10%   0%   18-­‐34   35-­‐44   45-­‐54   55-­‐64   65+   [Nielsen2011]   [USCensusBureau2008]  
  45. 45. Collec@ng  informa@on  Objec@ve  data   –  As  early  as  possible  [Henze2011IJMHCI]   –  More  than  just  the  task  performance   •  All  aspects  that  affect  the  results   •  E.g.  device  type,  local,  @me,  screen  size,  resolu@on,  ...   •  In  par@cular:  a  version  number   –  Compromise  between  permissions  and  data  to   collect    
  46. 46. Collec@ng  informa@on  Subjec@ve  data   –  App  Store  comments  can  provide  informa@on   •  but  usually  dont  [Henze2011IJMHCI]   •  Might  help  to  claim  an  app  is  great  (e.g.  [Zhai2009CHI])   •  Ra@ngs  without  baseline  are  meaningles   –  Inves@gated  how  to  get  subjec@ve  feedback   [McMillan2010Pervasive]   •  In-­‐game  “tasks”  with  dynamically  loaded  ques@ons   •  Integra@on  with  Facebook   •  Interviewed  10  people  over  VoIP  for  $25  
  47. 47. Realy  stupid     hope   Stupid  waste  of  @me!!!   cailan     FC  the  rabbit....  uninstalled   Godimus  Prime     Ready  for  prime  Its  ok   Stupid  waste  of  @me.   erika   lance   @me   boring  and  dumb.    Users  don’t  care  if  it’s  a   Beba   research  prototype   Stupid  and  offincive  to  my  pet  rabbit  bayleigh   Logan   1  word......  dumb!   josue   5  stars  if  there  is  a  way  to  turn  the  music  off.   Doesnt  go  to  well  with  slipknot   Allen   What  the  hell  is  this??   Boo!   Luci   Cullen  Girl   Examples  from  one  of  my   games  
  48. 48.      Ready  for  prime   @me    Users  don’t  care  if  it’s  a   research  prototype    Low  quality  results  in  low   ra@ngs  
  49. 49.      Ready  for  prime   @me    users  don’t  care  if  it’s  a   research  prototype    low  quality  results  in  low   ra@ngs    and  few  install   installa@ons  
  50. 50. Ethical  and  legal  issues  “One  should  treat  others  as  one  would  like  others  to  treat  oneself”  [Flew1979Dic@onary]   “Primum  non  nocere”/”First,  do  no   harm”  (Thomas  Sydenham)  
  51. 51. 6.96%   57.28%         Informed  consent    Presenta@on  highly  affects   the  conversion  rate   67.42%   87.57%  [Pielot2011ELV]  
  52. 52.       Informed  consent    Presenta@on  highly  affects   the  conversion  rate    Par@cipants  arent  aware   what  data  is  collected  [Morrison2011CHI]  
  53. 53.      Regula@ons    Which  rules  to  follow?  
  54. 54.    “any   informa+on   rela+ng   to   an   iden+fied    or  iden+fiable  natural  person”   Regula@ons   •  Transparency:  the  persons  whose  data    Which  rules  to  follow?   are  being  collected  or  accessed  have  the   right   to   be   informed   when   such   data    e.g.  EU  Data  Protec@on   processing  is  taking  place.   Direc@ve   •  Legi+mate   purpose:   data   can   only   be   collected  for  specific  purposes   •  Propor+onality :   data   should   be   processed   in   a   fashion   that   is   not   excessive  beyond  the  purposes  for  which   they  were  collected   [Henderson2009HotPlanet]  
  55. 55. …  or  what  works  for  me  
  56. 56.   number  of  installa+ons     400     350   Games  vs.  Apps  Thousands   300    our  games  are  more   250   successful   200   150   100   50   0  
  57. 57.     games   15,6%     Games  vs.  Apps    our  games  are  more  successful    there  are  more  apps  than   games   apps   84,4%   available  in  the  Android  Market  hXp://  
  58. 58.      Games  vs.  Apps    our  games  are  more  successful    there  are  more  apps  than   games    players  execute  the   strangest  tasks      
  59. 59.      Games  vs.  Apps    our  games  are  more  successful    there  are  more  apps  than   games    players  execute  the   strangest  tasks    widgets  and  background   services  are  perfect  for   longitudinal  observa@ons      
  60. 60.      Informing  the  user    provide  informa@on  in  the   Market      
  61. 61.      Informing  the  user    provide  informa@on  in  the   Market    show  a  modal  dialog  at  the   first  start      
  62. 62.      Informing  the  user    provide  informa@on  in  the   Market    show  a  modal  dialog  at  the   first  start    provide  more  informa@on   and  a  link  in  an  about  page      
  63. 63.      Publishing    fancy  screenshots  and  icon   (that  the  first  thing   someone  sees)    @tle  &  descrip@on  contain   words  users  search  for    of  course  I  don’t  want  to   miss  a  single  user    prepare  a  dedicated   webpage  for  each  app      
  64. 64.      Playing  with  the   market    weekly  updates      
  65. 65.      Playing  with  the   market    weekly  updates    rate  your  app  as  soon  as  it   becomes  available  (and   force  everyone  you  know   to  do  it  as  well)      
  66. 66.      Keep  it  simple    focused  and  specialized   studies      
  67. 67.      Keep  it  simple    focused  and  specialized   studies    learning  by  doing      
  68. 68.      Keep  it  simple    focused  and  specialized   studies    learning  by  doing    release  early,  o]en,  and   try  it  again  if  it  doesn’t   work  
  69. 69.      Logging    use  hXp  to  transmit  data      
  70. 70.      Logging    use  hXp  to  transmit  data    store  unaggregated  measures      
  71. 71.       Logging    use  hXp  to  transmit  data    store  unaggregated  measures    consider  limited  resources       in  total:   392,401      files  27,331,383,646  bytes   Examples  from  one  of  my   games  
  72. 72.         How  to  do  Mobile     HCI  Research  in   Wrap  up   the  large?       ethnography,  controlled   Niels  Henze  experiments,  observa@ons,     …  can  all  work  in  the  large     University  of  Oldenburg   Media  Informa@cs  and     collect  data  early,            Mul@media  Systems       release  o]en,  be  flexible     OFFIS  -­‐  Ins@tute  for  Informa@on   respect  ethics,            Technology   ser  Interface  Group               Intelligent  U consider  regula@ons    
  73. 73. References  [Wang2009NIME]  Ge  Wang:  Designing  Smule’s  iPhone  Ocarina.  Proc.  NIME,  2009.  [Zhai2009CHI]  Zhai,  S.,  Kristensson,  P.O.,  Gong,  P.,  Greiner,  M.,  Peng,  S.,  Liu,  L.  Dunnigan,  A.,  Shapewriter  on  the  iPhone:   from  the  laboratory  to  the  real  world.  Adjunct  Proc.  CHI,  2009.  [Gilbertson2008CiE]  Paul  Gilbertson,  Paul  Coulton,  Fadi  Chehimi,  Tamas  Vajk:  Using  Tilt  as  an  Interface  to  control  No   BuXon  3-­‐D  Mobile  Games.  ACM  Computers  in  Entertainment,  2008.  [Oliver2010HotPlanet]  Earl  Oliver.  The  Challenges  in  Large-­‐Scale  Smartphone  User  Studies.  Invited  talk  @  HotPlanet,   2010.  [McMillan2010RiL]  Donald  McMillan:  iPhone  So]ware  Distribu@on  for  Mass  Par@cipa@on.  Proc.  Research  in  the  Large   Workshop  @  UbiComp,  2010.  [Miluzzo2010RiL]  Emiliano  Miluzzo,  Nicholas  D.  Lane,  Hong  Lu,  Andrew  T.  Campbell:  Research  in  the  App  Store  Era:   Experiences  from  the  CenceMe  App  Deployment  on  the  iPhone.  Proc.  Research  in  the  Large  Workshop  @   UbiComp,  2010.  [Henze2011IJMHCI]  Niels  Henze,  Mar@n  Pielot,  Benjamin  Poppinga,  Torben  Schinke,  Susanne  Boll:  My  App  is  an   Experiment:  Experience  from  User  Studies  in  Mobile  App  Stores,  accepted  by  the  Interna@onal  Journal  of   Mobile  Human  Computer  Interac@on  (IJMHCI),  2011  [McMillan2010Pervasive]  Donald  McMillan,  Alistair  Morrison,  Owain  Brown,  Malcolm  Hall  &  MaXhew  Chalmers:   Further  into  the  Wild:  Running  Worldwide  Trials  of  Mobile  Systems,  Proc.  Pervasive  2010.  [Cramer2010UbiComp]  HenrieXe  Cramer,  Mahas  Rost,  Nicolas  Belloni,  Didier  Chincholle,  Frank  Bentley:  Research  in   the  Large.  Using  App  Stores,  Markets,  and  Other  Wide  Distribu@on  Channels  in  Ubicomp  Research.  Adjunct   Proc.  Ubicomp,  2010.  [Morrison2010RiL]  Alistair  Morrison,  Stuart  Reeves,  Donald  McMillan,  MaXhew  Chalmers:  Experiences  of  Mass   Par@cipa@on  in  Ubicomp  Research,  Proc.  Research  In  The  Large  Workshop  at  Ubicomp,  2010.  [Poppinga2010OMUE]  Benjamin  Poppinga,  Mar@n  Pielot,  Niels  Henze,  Susanne  Boll:  Unsupervised  User  Observa@on  in   the  App  Store:  Experiences  with  the  Sensor-­‐based  Evalua@on  of  a  Mobile  Pedestrian  Naviga@on  Applica@on.   Proc.  OMUE  in  conjunc@on  with  NordiCHI,  2010.  
  74. 74. References  [Pielot2011ELV]  Mar@n  Pielot,  Niels  Henze,  Susanne  Boll:  Experiments  in  App  Stores  –  How  to  Ask  Users  for  their   Consent?,  Proceedings  of  the  CHI  workshop  on  Ethics,  logs  &  videotape,  2011.  [Henderson2009HotPlanet]  Tristan  Henderson,  Fehmi  Ben  Abdesslem:  Scaling  Measurement  Experiments  to  Planet-­‐ Scale:  Ethical,  Regulatory  and  Cultural  Considera@ons.  Proc.  HotPlanet,  2009.  [Morrison2011CHI]  Alistair  Morrison,  Owain  Brown,  Donald  McMillan,  MaXhew  Chalmers:  Informed  Consent  and   Users  Ahtudes  to  Logging  in  Large  Scale  Trials.  Adjunct  Proc.  CHI,  2011.  [Norcie2011ELV]  Greg  Norcie:  Ethical  and  Prac@cal  Considera@ons  For  Compensa@on  of  Crowdsourced  Research   Par@cipants,  Proc.  ETHICS,  LOGS  and  VIDEOTAPE  @  CHI,  2011.  [Girardello2010DSZ]  A.  Girardello,  F.  Michahelles,  Explicit  and  Implicit  Ra@ngs  for  Mobile  Applica@ons.  In  3.  Workshop   “Digitale  Soziale  Netze”  and  der  40.  Jahrestagung  der  Gesellsha]  für  Informa@k,  September  2010,  Leipzig.  [Riccamboni2010IB]  Rodolfo  Riccamboni,  Alessio  Mereu,  Chiara  Boscarol:  Keys  to  Nature:  A  test  on  the  iPhone  market.   Tools  for  Iden@fying  Biodiversity:  Progress  and  Problems,  2010.  [Kuhn2010MM]  Michael  Kuhn,  Roger  WaXenhofer,  Samuel  Welten:  Social  Audio  Features  for  Advanced  Music  Retrieval   Interfaces.  Proc.  MM,  2010.  [Yan2011MobiSys]Bo  Yan,  Guanling  Chen:  AppJoy:  Personalized  Mobile  Applica@on  Discovery.  Proc.  MobiSys,  2011.  [Budde2010IoT]  Andreas  Budde,  Florian  Michahelles:  Product  Empire  -­‐  Serious  play  with  barcodes.  Proc.  IoT,  2010.  [Karpischek2011RiL]  Stephan  Karpischek,  Geron  Gilad,  Florian  Michahelles:  Towards  a  BeXer  Understanding  of  Mobile   Shopping  Assistants  -­‐  A  Large  Scale  Usage  Analysis  of  a  Mobile  Bargain  Finder  Applica@on.  Workshop  on   Research  in  the  Large  @  UbiComp,  2011.  [Henze2010MobileHCI]  Niels  Henze,  Susanne  Boll:  Push  the  Study  to  the  App  Store:  Evalua@ng  Off-­‐Screen  Visualiza@ons   for  Maps  in  the  Android  Market,  Proc.  MobileHCI,  2010  [Henze2010NordiCHI]  Niels  Henze,  Benjamin  Poppinga,  Susanne  Boll:  Experiments  in  the  Wild:  Public  Evalua@on  of  Off-­‐ Screen  Visualiza@ons  in  the  Android  Market,  Proc.  NordiCHI,  2010.  
  75. 75. References  [Hood2011IJTR]  Jeffrey  Hood,  Elizabeth  Sall,  Billy  Charlton:  A  GPS-­‐based  Bicycle  Route  Choice  Model  for  San  Francisco,   California.  Transporta@on  LeXers:  The  Interna@onal  Journal  of  Transporta@on  Research,  2011  [Henze2011MobileHCIa]  Niels  Henze,  Enrico  Rukzio,  Susanne  Boll:  100,000,000  Taps:  Analysis  and  Improvement  of   Touch  Performance  in  the  Large,  Proceedings  of  MobileHCI,  2011  [Henze2011MobileHCIbNiels  Henze,  Susanne  Boll:  Release  Your  App  on  Sunday  Eve:  Finding  the  Best  Time  to  Deploy   Apps,  Adjunct  proceedings  of  MobileHCI,  2011  [Watzdorf2010LocWeb]  Stephan  von  Watzdorf,  Florian  Michahelles:  Accuracy  of  Posi@oning  Data  on  Smartphones.   Proc.  LocWeb,  2010.  [Ferreira2011Pervasive]  Denzil  Ferreira,  Anind  K.  Dey,  Vassilis  Kostakos:  Understanding  Human-­‐Smartphone  Concerns:   A  Study  of  BaXery  Life.  Proc.  Pervasive,  2011.  [Buddharaju2010CHI]  Pradeep  Buddharaju,  Yuichi  Fujiki,  Ioannis  Pavlidis,  Ergun  Akleman:  A  Novel  Way  to  Conduct   Human  Studies  and  Do  Some  Good.  Adcunct  Proc.  CHI,  2010.  [Sahami2011CHI]  Alireza  Sahami,  Michael  Rohs,  Robert  Schleicher,  Sven  Kratz,  Alexander  Müller,  Albrecht  Schmidt:   Real-­‐Time  Nonverbal  Opinion  Sharing  through  Mobile  Phones  during  Sports  Events,  Proc.  CHI  2011.  [Verkasalo2010MB]  Hannu  Verkasalo:  Analysis  of  Smartphone  User  Behavior,  Proc.  Ninth  Interna@onal  Conference  on   Mobile  Business,  2010.  [Böhmer2011MobileHCI]  MaXhias  Böhmer,  Brent  Hecht,  Johannes  Schöning,  Antonio  Krüger,  Gernot  Bauer:  Falling   Asleep  with  Angry  Birds,  Facebook  and  Kindle  –  A  Large  Scale  Study  on  Mobile  Applica@on  Usage.  Proc.   MobileHCI,  2011.  [Agarwal2010HotNets]  Sharad  Agarwal,  Ratul  Mahajan,  Alice  Zheng,  Victor  Bahl:  There’s  an  app  for  that,  but  it  doesn’t   work.  Diagnosing  Mobile  Applica@ons  in  the  Wild.  Proc.  Hotnets,  2010.  [Morrison2010RiL]  Alistair  Morrison,  MaXhew  Chalmers:  SGVis:  Analysis  of  Mass  Par@cipa@on  Trial  Data.  Proc.   Research  In  The  Large  Workshop  at  Ubicomp,  2010.  [Lane2010CM]  Nicholas  D.  Lane,  Emiliano  Miluzzo,  Hong  Lu,  Daniel  Peebles,  Tanzeem  Choudhury,  Andrew  T.  Campbell:   A  Survey  of  Mobile  Phone  Sensing.  IEEE  Communica@ons  Magazine,  2010.