Online Multitasking and User Engagement

254 Aufrufe

Veröffentlicht am

Presentation at CIKM, held in San Francisco, October 2013.

Users often access and re-access more than one site during an online session, effectively engaging in multitasking. In this work, we study the effect of online multitasking on two widely used engagement metrics designed to capture users browsing behavior with a site. Our study is based on browsing data of 2.5M users across 760 sites encompassing diverse types of services such as social media, news and mail. To account for multitasking we need to redefine how user sessions are represented and we need to adapt the metrics under study. We introduce a new representation of user sessions: tree-streams – as opposed to the commonly used click-streams – present a more accurate picture of the browsing behavior of a user that includes how users switch between sites (e.g., hyperlinking, teleporting, backpaging). We then discuss a number of insights on multitasking patterns, and show how these help to better understand how users engage with sites. Finally, we define metrics that characterize multitasking during online sessions and show how they provide additional insights to standard engagement metrics.

This work was done in collaboration with Mounia Lalmas, George Dupret, and Ricardo Baeza-Yates.

Veröffentlicht in: Daten & Analysen
0 Kommentare
1 Gefällt mir
Statistik
Notizen
  • Als Erste(r) kommentieren

Keine Downloads
Aufrufe
Aufrufe insgesamt
254
Auf SlideShare
0
Aus Einbettungen
0
Anzahl an Einbettungen
1
Aktionen
Geteilt
0
Downloads
6
Kommentare
0
Gefällt mir
1
Einbettungen 0
Keine Einbettungen

Keine Notizen für die Folie

Online Multitasking and User Engagement

  1. 1. ONLINE MULTITASKING AND USER ENGAGEMENT CIKM 2013 In  collabora*on  with:   Mounia  Lalmas,     Ricardo  Baeza-­‐Yates,     George  Dupret   Jane%e  Lehmann  
  2. 2. outline 1.  Mo%va%on   How  do  users  browse  the  web  today?       2.  Characteris%cs  of  online  mul%tasking   Ac2vity  during  and  between  visits       3.  Measuring  online  mul%tasking   Defini2on  of  new  metrics,  case  study     Lights  on  by  JC*+A!  
  3. 3. How  do  users   browse  the  Web   today?   leC  by    [  embr  ]    
  4. 4. ONLINE MULTITASKING 4  JaneGe  Lehmann   Mo2va2on   Browsing  the  “old  way”   facebook   news   news  news   news   mail   1min   2min   1min   3min   Dwell  2me  during  a  visit  on  a  news  site:   7min  on  average   news  site  
  5. 5. ONLINE MULTITASKING 5  JaneGe  Lehmann   Mo2va2on   Nowadays   news   facebook   mail  news   news   news   1min   2min   1min   3min   Dwell  2me  during  a  visit  on  a  news  site:   2.33min  on  average  (1min  |  3min  |  3min)  
  6. 6. ONLINE MULTITASKING 6  JaneGe  Lehmann   Mo2va2on   •  Users  switch  between  sites,  to  do  related  or  totally  unrelated  tasks         •  E.  Herder  [1]:   »  75%  of  sites  are  visited  more  than  once   »  74%  of  revisits  are  performed  within  a  session     Measuring  browsing  behavior  can  lead  to  incorrect  conclusions.     [1]  E.  Herder.  Characteriza*ons  of  user  web  revisit  behavior.  In  LWA,  2005.  
  7. 7. Characteris%cs   of  online   mul%tasking   Danboard's  Messy  Home  by  Mullenkedheim  
  8. 8. DATA SET Interac%on  data   •  July  2012   •  2.5M  users   •  785M  page  views     •  We  defined  a  new  naviga2on  model                                             (see  paper  for  detail)       •  Categoriza2on  of  the  most  frequent  accessed  sites   (e.g.  mail,  news,  shopping)   »  11  categories  (news),  33  subcategories  (e.g.  news   finance,  news  society)   »  760  sites  from  70  countries/regions       8  JaneGe  Lehmann   Characteris2cs  
  9. 9. Visit activity Visit  frequency     9  JaneGe  Lehmann   Characteris2cs   Mul%tasking  depends  on  the  site  under   considera%on     •  Social  media  sites  are  revisited  the   most   •  News  (tech)  sites  are  the  least     revisited  sites   news (finance) news (tech) social media mail 2.09 1.76 2.28 2.09 4.65 1.59 4.78 4.61 #Visits (avg sd)
  10. 10. Visit activity Ac%vity  between  visits       10  JaneGe  Lehmann   Characteris2cs   Differences  in  the  absence  %me     •  50%  of  sites  are  revisited  aCer  less   than  1min            -­‐  Interrup*on  of  a  task   •  There  are  revisits  aCer  a  long  break               -­‐  Returning  to  a  site  to  perform  a  new   task   0.00 0.25 0.50 0.75 1.00 10 2 10 1 100 101 102 mail social media news (finance) news (tech) Cumulativeprobability Absence time [min] *   v2  v1   *   v3   *  -­‐  absence  2me  
  11. 11. Visit activity Ac%vity  paLern       11  JaneGe  Lehmann   Characteris2cs   •  Four  types  of  "aGen2on  shiCs”   •  Complex  cases  refer  to  no   specific  paGern  or  repeated   paGern   •  Successive  visits  can  belong   together  (i.e.,  to  the  same  task)   0.23 0.28 0.33 mail sites news (finance) sites news (tech) sites social media sites decreasing attention increasing attention constant attention complex attention Proportionoftotal dwelltimeonsite p-value = 0.09 m = -0.01 p-value = 0.07 m = -0.02 p-value = 0.79 m = 0.00 0.23 0.28 0.33 Proportionoftotal dwelltimeonsite
  12. 12. Danboard  by  sⓘndy°   Measuring     online   mul%tasking    
  13. 13. Cumulative activity Cumula%ve  ac%vity                            vi  Browsing  ac2vity  during  the  ith  visit                    ivi  Browsing  ac2vity  between  the  (i-­‐1)th  and  ith  visit                    k=3  Rescaling  factor  for  ivi                    m  Browsing  ac2vity  (e.g.  dwell  2me,  page  views)     Assump%on:   If  users  return  aCer  short  2me,  they  return  to  con2nue  with  same  task.   If  users  return  aCer  longer  2me,  they  return  to  perform  a  new  task  -­‐  an  indica2on  of  loyalty  to   the  site.     13  JaneGe  Lehmann   Metrics   CumActm,k = log10 (v1 + ivi k •vi i=2 n ∑ ) iv2   v2  v1   iv3   v3  
  14. 14. Cumulative activity Cumula%ve  ac%vity                            vi  Browsing  ac2vity  during  the  ith  visit                    ivi  Browsing  ac2vity  between  the  (i-­‐1)th  and  ith  visit                    k=3  Rescaling  factor  for  ivi                    m  Browsing  ac2vity  (e.g.  dwell  2me,  page  views)     Assump%on:   If  users  return  aCer  short  2me,  they  return  to  con2nue  with  same  task.   If  users  return  aCer  longer  2me,  they  return  to  perform  a  new  task  -­‐  an  indica2on  of  loyalty  to   the  site.     14  JaneGe  Lehmann   Metrics   CumActm,k = log10 (v1 + ivi k •vi i=2 n ∑ ) iv2   v2  v1   iv3   v3  
  15. 15. Cumulative activity Cumula%ve  ac%vity                            vi  Browsing  ac2vity  during  the  ith  visit                    ivi  Browsing  ac2vity  between  the  (i-­‐1)th  and  ith  visit                    k=3  Rescaling  factor  for  ivi                    m  Browsing  ac2vity  (e.g.  dwell  2me,  page  views)     Assump%on:   If  users  return  aCer  short  2me,  they  return  to  con2nue  with  same  task.   If  users  return  aCer  longer  2me,  they  return  to  perform  a  new  task  -­‐  an  indica2on  of  loyalty  to   the  site.     15  JaneGe  Lehmann   Metrics   CumActm,k = log10 (v1 + ivi k •vi i=2 n ∑ ) iv2   v2  v1   iv3   v3   v1  +  v2  +  v3      
  16. 16. Cumulative activity Cumula%ve  ac%vity                            vi  Browsing  ac2vity  during  the  ith  visit                    ivi  Browsing  ac2vity  between  the  (i-­‐1)th  and  ith  visit                    k=3  Rescaling  factor  for  ivi                    m  Browsing  ac2vity  (e.g.  dwell  2me,  page  views)     Assump%on:   If  users  return  aCer  short  2me,  they  return  to  con2nue  with  same  task.   If  users  return  aCer  longer  2me,  they  return  to  perform  a  new  task  -­‐  an  indica2on  of  loyalty  to   the  site.     16  JaneGe  Lehmann   Metrics   CumActm,k = log10 (v1 + ivi k •vi i=2 n ∑ ) iv2   v2  v1   iv3   v3  
  17. 17. Cumulative activity Cumula%ve  ac%vity                            vi  Browsing  ac2vity  during  the  ith  visit                    ivi  Browsing  ac2vity  between  the  (i-­‐1)th  and  ith  visit                    k=3  Rescaling  factor  for  ivi                    m  Browsing  ac2vity  (e.g.  dwell  2me,  page  views)     Assump%on:   If  users  return  aCer  short  2me,  they  return  to  con2nue  with  same  task.   If  users  return  aCer  longer  2me,  they  return  to  perform  a  new  task  -­‐  an  indica2on  of  loyalty  to   the  site.     17  JaneGe  Lehmann   Metrics   CumActm,k = log10 (v1 + ivi k •vi i=2 n ∑ ) iv2   v2  v1   iv3   v3   v1  +  (iv2)3Ÿ  v2  +  (iv3)3Ÿ  v3      
  18. 18. Activity pattern ALen%on  shiN  and  range                            n=4    Number  of  visits  in  session                      σ  Variance  in  the  visit  ac2vity                    μ  Average  of  the  visit  ac2vity                    inv  Modifica2on  of  the  “Inversion  number”         Descrip%on:   AGShiC  models  the  shiC  of  aGen2on  in  the  browsing  ac2vity   AGRange  describes  fluctua2ons  in  the  browsing  ac2vity     18  JaneGe  Lehmann   Metrics   AttShiftm,n = invm,n − minInvm,n | maxInvm,n |− | minInvm,n | AttRangem,n = σ (Vm,n ) µ(Vm,n )
  19. 19. Activity pattern ALen%on  shiN  and  range   19  JaneGe  Lehmann   Metrics   -­‐1   0   1   0   constant   constant   constant   >  0   decreasing   complex   increasing   AUen*on  shiV   AUen*on  range  
  20. 20. Comparing  the  ranking  of  the  sites   •  Visitdt  –  Dwell  2me  during  a  visit   •  Sessiondt  –  Dwell  2me  during  a  session                     Ø  Visitdt  and  Sessiondt  correlate   Ø  Otherwise  no  correla2on  à  the  other  metrics  capture  different  aspects  of   browsing  behavior   Comparing metrics 20  JaneGe  Lehmann   Metrics   Visitdt   Sessiondt   CumActdt   ALShiNdt   Sessiondt   0.57   CumActdt   -­‐0.04   0.24   ALShiNdt   0.09   0.22   0.02   ALRangedt   -­‐0.01   -­‐0.01   -­‐0.26   0.19  
  21. 21. “Models”  of  browsing  behavior   •  Clustering  of  sites  using  mul2tasking  and  standard  engagement  metrics:   •  CumActdt,  AGShiCdt,  AGRangedt   •  Visitdt,  Sessiondt     •  We  iden2fied  five  cluster:                 Models of browsing behavior 21  JaneGe  Lehmann   Metrics   C4: 74 sites 0.25 -0.25 0.75 -0.75 C5: 166 sites 0.25 -0.25 0.75 -0.75 C3: 156 sites 0.25 -0.25 0.75 -0.75 C2: 108 sites 0.25 -0.25 0.75 -0.75 C1: 172 sites 0.25 -0.25 0.75 -0.75 Visitdt [min] CumActdt,3 AttShiftdt,4 AttRangedt,4 Sessiondt [min]
  22. 22. Models of browsing behavior 22  JaneGe  Lehmann   Metrics   Visitdt [min] CumActdt,3 AttShiftdt,4 AttRangedt,4 Sessiondt [min] C2: 108 sites auctions, front page, shopping, dating 0.25 -0.25 0.75 -0.75 C1: 172 sites mail, maps, news, news (soc.) 0.25 -0.25 0.75 -0.75 One  task  during  a  session     §  High  dwell  2me  per  visit  and  during   the  whole  session     §  Users  return  to  con2nue  a  task  (short   absence  2me)     §  C1:  aGen2on  is  shiCing  to  another  site   §  C2:  aGen2on  is  shiCing  slowly  towards   the  site  
  23. 23. C4: 74 sites front page, search, download C3: 156 sites auctions, search, front page, shopping 0.25 -0.25 0.75 -0.75 0.25 -0.25 0.75 -0.75 Models of browsing behavior 23  JaneGe  Lehmann   Metrics   Several  tasks  during  a  session     §  Users  perform  several  tasks  on  these   sites  during  a  session   §  No  simple  ac2vity  paGern     §  C3:  Dwell  2me  per  visit  is  low,  but  the   dwell  2me  per  session  is  high     Visitdt [min] CumActdt,3 AttShiftdt,4 AttRangedt,4 Sessiondt [min]
  24. 24. C5: 166 sites service, download, blogging, news (soc.) 0.25 -0.25 0.75 -0.75 Models of browsing behavior 24  JaneGe  Lehmann   Metrics   Sites  with  low  ac%vity     §  Users  do  not  spend  a  lot  of  2me  on   these  sites     §  Time  between  visits  is  short     §  AGen2on  is  shiCing  towards  the  site   Visitdt [min] CumActdt,3 AttShiftdt,4 AttRangedt,4 Sessiondt [min]
  25. 25. C2: 108 sites auctions, front page, shopping, dating 0.25 -0.25 0.75 -0.75 C3: 156 sites auctions, search, front page, shopping 0.25 -0.25 0.75 -0.75 Models of browsing behavior 25  JaneGe  Lehmann   Metrics   Browsing  behavior  can  differ  between   sites  of  the  same  category     §  C2:  users  visit  site  once  to  perform   their  task   §  C3:  users  visit  site  several  2mes  to   perform  task(s)   Visitdt [min] CumActdt,3 AttShiftdt,4 AttRangedt,4 Sessiondt [min]
  26. 26. SUMMARY and Future Work JaneGe  Lehmann   26   •  Online  mul2tasking  affects  the  way  users  access  sites  –  Standard  metrics   do  not  capture  this!!!   •  We  defined  metrics  that  describe  different  aspects  of  mul2tasking   •  CumAct  accounts  for  the  2me  between  visits   •  AGShiC,  AGRange  describe  aGen2on  shiCs   •  We  showed  that  mul2tasking  depends  on  the  site  under  considera2on     Future  work:   •  Can  we  improve  the  defini2on  of  a  task?   •  How  does  mul2tasking  affect  other  metrics,  such  as  bounce  rate  and  click-­‐ through  rate?   •  Does  mul2tasking  differ  in  different  countries?   Summary  
  27. 27. Janette Lehmann Universitat Pompeu Fabra, Spain lehmannj@acm.org Mounia Lalmas Yahoo Labs London mounia@acm.org George Dupret Yahoo Labs Sunnyvale gdupret@yahoo-inc.com Ricardo Baeza-Yates Yahoo Labs Barcelona rbaeza@acm.org Online Multitasking + User Engagement

×