photo	
  credit	
  donsolo,	
  CC	
  BY-­‐NC-­‐SA	
  2.0	
  
From	
  Site	
  to	
  Inter-­‐site	
  User	
  Engagement	
  
...
•  User	
  engagement	
  is	
  a	
  quality	
  of	
  the	
  user	
  experience	
  that	
  emphasizes	
  the	
  
posiLve	
 ...
User	
  Engagement	
  
Measuring	
  
3	
  IntroducLon	
  
Before	
  we	
  can	
  design	
  engaging	
  websites,	
  
it	
 ...
Main	
  Research	
  Goals	
  
4	
  IntroducLon	
  
Primary	
  goal 	
   	
  Can	
  we	
  define	
  new	
  engagement	
  met...
IntroducLon	
   5	
  
Analysis/
Planning	
  
Design	
  Changes	
  Measuring	
  Online	
  mulLtasking	
  
Inter-­‐site	
  
...
IntroducLon	
   6	
  
Analysis/
Planning	
  
Design	
  Changes	
  Measuring	
  
Site	
  engagement	
  
Measuring	
  Engagement	
  
InteracLon	
  data	
  
7	
  Site	
  engagement	
  
Data	
  
Browsing	
  events	
  provided	
  ...
Site	
  engagement	
   8	
  
Measuring	
  Engagement	
  
Online	
  behaviour	
  metrics	
  
K.	
  Rodden,	
  H.	
  Hutchin...
Site	
  engagement	
   9	
  
Measuring	
  Engagement	
  
Online	
  behaviour	
  metrics	
  
Popularity	
  
#Users	
   Numb...
Site	
  engagement	
   10	
  
Measuring	
  Engagement	
  
Differences	
  in	
  engagement	
  
ComScore,	
  Alexa,	
  
Googl...
Measuring	
  Engagement	
  
Problem	
  
11	
  Site	
  engagement	
  
Isolated	
  view:	
  The	
  metrics	
  focus	
  
on	
...
IntroducLon	
   12	
  
Analysis/
Planning	
  
Design	
  Changes	
  Measuring	
  Online	
  mulLtasking	
  
Site	
  engageme...
Online	
  mulLtasking	
   13	
  
MoCvaCon	
  
In-­‐the-­‐moment	
  engagement	
  
ComScore,	
  Alexa,	
  
GoogleAnalyHcs,…...
Online	
  mulLtasking	
   14	
  
MoCvaCon	
  
In-­‐the-­‐moment	
  engagement	
  
ComScore,	
  Alexa,	
  
GoogleAnalyHcs,…...
Online	
  mulLtasking	
   15	
  
MoCvaCon	
  
Online	
  mulLtasking.	
  
Problem	
  
	
  
•  Engagement	
  metrics	
  do	
...
Research	
  QuesCon	
  
16	
  Online	
  mulLtasking	
  
How	
  can	
  we	
  measure	
  engagement	
  by	
  
accounLng	
  f...
Extent	
  of	
  mulCtasking	
  
•  10.2	
  disLnct	
  sites,	
  2	
  visits	
  per	
  site.	
  
	
  
Absence	
  Cme	
  
• ...
AcCvity	
  paPerns	
  	
  
•  Four	
  types:	
  Decreasing,	
  increasing,	
  constant,	
  complex.	
  
•  Successive	
  v...
Online	
  mulLtasking	
   19	
  
Measuring	
  Engagement	
  
Online	
  mulLtasking	
  metrics	
  
Extent	
  of	
  mulCtask...
20	
  
CASE	
  STUDY:	
  
MulCtasking	
  PaPerns	
  
	
  
•  ObjecCve:	
  Analyse	
  mulLtasking	
  acLvity	
  on	
  sites...
21	
  
Case	
  Study:	
  MulCtasking	
  PaPerns	
  
Results	
  
No	
  mulCtasking	
   MulCtasking	
  
Quick	
   Focused	
 ...
22	
  
Case	
  Study:	
  MulCtasking	
  PaPerns	
  
Results	
  
No	
  mulCtasking	
   MulCtasking	
  
Quick	
   Focused	
 ...
23	
  
CASE	
  STUDY:	
  
Wikipedia	
  (on-­‐site	
  mulCtasking)	
  
	
  
•  ObjecCve:	
  Analyse	
  reading	
  acLvity	
...
24	
  
Case	
  Study:	
  Wikipedia	
  
Approach	
  
Online	
  mulLtasking	
  
Users’	
  reading	
  behaviour	
  on	
  an	
...
25	
  
Case	
  Study:	
  Wikipedia	
  
Results	
  
No	
  mulCtasking	
   MulCtasking	
  
Focus	
   ExploraCon	
   Passing	...
 
On-­‐site	
  mulCtasking	
  
	
  
•  MulLtasking	
  between	
  news	
  arLcles	
  of	
  a	
  provider.	
  
•  MulLtaskin...
Take	
  Aways	
  
•  AccounLng	
  for	
  mulLtasking	
  leads	
  to	
  a	
  
be;er	
  understanding	
  on	
  how	
  users	...
IntroducLon	
   28	
  
Analysis/
Planning	
  
Design	
  Changes	
  Measuring	
  Online	
  mulLtasking	
  
Inter-­‐site	
  ...
Inter-­‐site	
  engagement	
   29	
  
MoCvaCon	
  
Large	
  online	
  service	
  providers	
  
ComScore,	
  Alexa,	
  
Goo...
Inter-­‐site	
  engagement	
   30	
  
MoCvaCon	
  
Large	
  online	
  service	
  providers	
  
frontpage	
  
tv	
   sports...
Inter-­‐site	
  engagement	
   31	
  
MoCvaCon	
  
Large	
  online	
  service	
  providers	
  
frontpage	
  
tv	
   sports...
Inter-­‐site	
  engagement	
   32	
  
MoCvaCon	
  
Online	
  mulLtasking	
  
Problem	
  
	
  
•  Engagement	
  metrics	
  ...
Research	
  QuesCon	
  
33	
  Inter-­‐site	
  engagement	
  
How	
  can	
  we	
  measure	
  
engagement	
  by	
  also	
  c...
Inter-­‐site	
  engagement	
   34	
  
Traffic	
  Networks	
  
Modelling	
  
We	
  model	
  sites	
  (nodes)	
  and	
  user	
...
Inter-­‐site	
  engagement	
   35	
  
Measuring	
  Engagement	
  
Inter-­‐site	
  engagement	
  metrics:	
  Network-­‐leve...
Inter-­‐site	
  engagement	
   36	
  
Measuring	
  Engagement	
  
Inter-­‐site	
  engagement	
  metrics:	
  Node-­‐level	
...
37	
  
CASE	
  STUDY:	
  
Yahoo	
  Provider	
  Networks	
  
	
  
•  ObjecCve:	
  Compare	
  networks;	
  characterise	
  t...
38	
  
Case	
  Study:	
  Yahoo	
  
Comparing	
  provider	
  networks	
  
Network	
  1	
   Network	
  2	
   Network	
  3	
 ...
39	
  
Case	
  Study:	
  Yahoo	
  
Sites	
  within	
  a	
  provider	
  network	
  
Traffic	
  Hub	
   Supporter	
  
Focused	...
 
Comparing	
  networks	
  
	
  
•  Device,	
  Lme,	
  upstream	
  traffic,	
  user.	
  
•  SimulaLons	
  (effect	
  of	
  ad...
Take	
  Aways	
  
•  Inter-­‐site	
  engagement	
  allows	
  for	
  a	
  
more	
  comprehensive	
  look	
  at	
  user	
  
...
IntroducLon	
   42	
  
Analysis/
Planning	
  
Design	
  Changes	
  Measuring	
  Online	
  mulLtasking	
  
Inter-­‐site	
  ...
43	
  
CASE	
  STUDY:	
  
Online	
  News	
  
	
  
•  Hypothesis:	
  It	
  may	
  be	
  beneficial	
  (long-­‐term)	
  to	
 ...
Types	
  of	
  reading	
  sessions	
  
	
  
No	
  click	
   	
  	
  	
  Did	
  not	
  follow	
  a	
  related	
  	
  	
  	
...
Providing	
  links	
  to	
  related	
  off-­‐site	
  content	
  has	
  a	
  
no	
  short-­‐term	
  effect,	
  but	
  a	
  po...
IntroducLon	
   46	
  
Analysis/
Planning	
  
Design	
  Changes	
  Measuring	
  Online	
  mulLtasking	
  
Inter-­‐site	
  ...
47	
  
CASE	
  STUDY:	
  
Yahoo	
  Provider	
  Network	
  
	
  
•  Hypothesis:	
  We	
  can	
  use	
  hyperlinks	
  to	
  ...
 
Hyperlink	
  vs.	
  traffic	
  network	
  
	
  
On-­‐site 	
   	
  Links/Traffic	
  to	
  pages	
  	
  
	
   	
   	
  within...
Hyperlinks	
  can	
  be	
  used	
  to	
  influence	
  site	
  
and	
  inter-­‐site	
  engagement	
  in	
  a	
  provider	
  ...
IntroducLon	
   50	
  
Analysis/
Planning	
  
Design	
  Changes	
  Measuring	
  Online	
  mulLtasking	
  
Inter-­‐site	
  ...
Two	
  new	
  perspecHves	
  for	
  measuring	
  
engagement	
  which	
  consider	
  the	
  
relaLonships	
  between	
  si...
AccounLng	
  for	
  the	
  new	
  perspecLves	
  
when	
  influencing	
  engagement.	
  
	
  
Online	
  news	
  
Providing	...
Wikipedia	
  
Providing	
  informaLon	
  about	
  readers’	
  
engagement	
  to	
  the	
  editor	
  community.	
  
Yahoo	
...
photo	
  credit	
  donsolo,	
  CC	
  BY-­‐NC-­‐SA	
  2.0	
  
Thank	
  you!	
  
	
  
Jane;e	
  Lehmann	
  
	
  
Barcelona,	...
•  Mounia Lalmas and Janette Lehmann. “Models of User Engagement”. In H. L. O’Brien and M. Lalmas (Eds.), Why Engagement
M...
User engagement
•  Mounia Lalmas, Heather L O’Brien, and Elad Yom-Tov. Measuring user engagement. Synthesis Lectures on Sa...
Online multitasking
•  Qing Wang and Huiyou Chang. Multitasking bar: prototype and evaluation of introducing the task conc...
Link economy
•  Joseph Turow and Lokman Tsui. The hyperlinked society. The University of Michigan Press, 2008.
•  Juliette...
ATTACHMENT:	
  
IntroducCon	
  
•  “In	
  a	
  world	
  full	
  of	
  choices	
  where	
  the	
  fleeCng	
  aPenCon	
  of	
  
the	
  user	
  becomes	
  a	
...
Methodology	
  
InteracLon	
  data,	
  online	
  sessions	
  and	
  site	
  visits.	
  
61	
  IntroducLon	
  
t0
t1
t2
t3
...
IntroducLon	
   62	
  
Thesis	
  structure	
  
Metrics that account for site
popularity, activity and loyalty
Advertising
...
ATTACHMENT:	
  
Site	
  engagement	
  
0-1 1-0.5 0.5
Kendall’s tau with p-value < 0.05
('-' insignificant correlations)
Site	
  engagement	
   64	
  
EvaluaCon	
...
Site	
  engagement	
   65	
  
PaPerns	
  of	
  Site	
  Engagement	
  
Engagement	
  depends	
  on	
  the	
  site	
  at	
  ...
ATTACHMENT:	
  
MulCtasking	
  
Online	
  mulLtasking	
   67	
  
MoCvaCon	
  
Users	
  switch	
  between	
  sites,	
  to	
  perform	
  related	
  or	
  to...
Online	
  mulLtasking	
   68	
  
Data	
  
Dataset	
  and	
  site	
  categories.	
  
Cat. Subcat. %Sites Description
news
2...
Online	
  mulLtasking	
   69	
  
MulCtasking	
  Metrics	
  
CumAct	
  accounts	
  for	
  the	
  acLvity	
  between	
  site...
Online	
  mulLtasking	
   70	
  
MulCtasking	
  Metrics	
  
AWRange	
  and	
  AWShik	
  describe	
  changes	
  between	
  ...
0-1 1-0.5 0.5
Spearman’s rho with p-value < 0.05
('-' insignificant correlations)
Online	
  mulLtasking	
   71	
  
EvaluaC...
Online	
  mulLtasking	
   72	
  
MulCtasking	
  PaPerns	
  
Cluster	
  centers,	
  site	
  categories	
  and	
  acLvity	
 ...
ATTACHMENT:	
  
Inter-­‐site	
  engagement	
  
Inter-­‐site	
  engagement	
   74	
  
Data	
  
Dataset,	
  network	
  and	
  site	
  categories.	
  
InteracCon	
  data	
 ...
Inter-­‐site	
  engagement	
   75	
  
Inter-­‐site	
  Engagement	
  Metrics	
  
Flow	
  accounts	
  for	
  the	
  extent	
...
Inter-­‐site	
  engagement	
   76	
  
Inter-­‐site	
  Engagement	
  Metrics	
  
Density	
  describes	
  the	
  connecLvity...
Inter-­‐site	
  engagement	
   77	
  
Inter-­‐site	
  Engagement	
  Metrics	
  
Reciprocity	
  measures	
  the	
  homogene...
Inter-­‐site	
  engagement	
   78	
  
Inter-­‐site	
  Engagement	
  Metrics	
  
Entry/ExitDisp	
  measures	
  how	
  the	
...
Inter-­‐site	
  engagement	
   79	
  
EvaluaCon:	
  Network-­‐level	
  
CorrelaLons	
  between	
  inter-­‐site	
  and	
  n...
Inter-­‐site	
  engagement	
   80	
  
EvaluaCon:	
  Node-­‐level	
  
CorrelaLons	
  between	
  inter-­‐site	
  and	
  site...
Inter-­‐site	
  engagement	
   81	
  
Comparing	
  Provider	
  Networks	
  
Country2
Country1
Country3
Country4
Country5
F...
Inter-­‐site	
  engagement	
   82	
  
PaPerns	
  of	
  Inter-­‐site	
  Engagement	
  CategoriesEngagement
PageRank EntryPr...
ATTACHMENT:	
  
NaCve	
  AdverCsing	
  
NaLve	
  AdverLsing	
   84	
  
Effect	
  on	
  User	
  Engagement	
  
0%
200%
400%
600%
short ad clicks long ad clicks
adcl...
NaLve	
  AdverLsing	
   85	
  
Mobile	
  vs.	
  Desktop	
  
Ad	
  post-­‐click	
  experience	
  between	
  mobile	
  
and	...
NaLve	
  AdverLsing	
   86	
  
Mobile	
  OpCmised	
  Landing	
  Pages	
  
Dwell	
  Cme:	
  The	
  distribuLon	
  is	
  ver...
ATTACHMENT:	
  
Wikipedia	
  
Wikipedia	
   88	
  
Wikipedia	
  Research	
  
Literature	
  review	
  by	
  Okoli	
  et	
  al.:	
  The	
  people’s	
  enc...
Wikipedia	
   89	
  
Reading	
  Preferences	
  
Popularitylow high
ArticleLengthshortlong
borderline casesII I
III IV
Jean...
Wikipedia	
   90	
  
Reading	
  PaPerns	
  Article
topic
Reading
behavior
ArticleViewsa
SessionArticlesa
Popularitya
Readi...
Wikipedia	
   91	
  
Reading	
  PaPerns	
  over	
  Time	
  
Stability	
  
•  30%	
  of	
  the	
  arLcles	
  are	
  popular...
ATTACHMENT:	
  
Yahoo	
  
93	
  
Upstream	
  Traffic	
  
TeleportaCon	
   Social	
  media	
  /	
  News	
   Search	
  /	
  Ext-­‐Yahoo	
  
Users	
  eng...
94	
  
Network	
  Effect	
  PaPern	
  
Yahoo	
  
Sites	
  change	
  their	
  popularity	
  in	
  the	
  same	
  way.	
  
Ac...
95	
  
Hyperlink	
  Performance	
  
Yahoo	
  
0%
25%
50%
75%
100%
Onsitelinks
front page providerservice news leisure
Inte...
ATTACHMENT:	
  
Online	
  News	
  
Online	
  news	
   97	
  
Focused	
  versus	
  Non-­‐focused	
  Sessions	
  
Internal	
  
Non-focused sessionsFocused sess...
Online	
  news	
   98	
  
Hyperlink	
  Performance	
  
	
  
Number	
  of	
  Inline	
  Links	
  
•  <10	
  links	
  may	
  ...
Online	
  news	
   99	
  
Effect	
  on	
  User	
  Engagement	
  
Internal	
  Focused	
  
	
  
Short-­‐term:	
  Only	
  3	
 ...
Online	
  news	
   100	
  
Effect	
  on	
  User	
  Engagement	
  
External	
  Focused	
  
	
  
Short-­‐term:	
  We	
  do	
 ...
Online	
  news	
   101	
  
Discovering	
  Story-­‐related	
  Content	
  in	
  TwiPer	
  
Nächste SlideShare
Wird geladen in …5
×

From Site to Inter-site User Engagement

524 Aufrufe

Veröffentlicht am

This is the presentation of my Ph.D. defense in February 2015.

The aim of my thesis is to provide a deeper understanding of how users engage with websites and how to measure this engagement. We start with studying online behaviour metrics, which are commonly employed as a proxy for user engagement, and we propose new metrics that expose so far unconsidered aspects of user engagement. We then conduct several case studies that demonstrate how these metrics provide a deeper understanding of user engagement. Within each case study we also examine how the characteristics of a website influence user engagement.

Veröffentlicht in: Daten & Analysen
0 Kommentare
1 Gefällt mir
Statistik
Notizen
  • Als Erste(r) kommentieren

Keine Downloads
Aufrufe
Aufrufe insgesamt
524
Auf SlideShare
0
Aus Einbettungen
0
Anzahl an Einbettungen
2
Aktionen
Geteilt
0
Downloads
8
Kommentare
0
Gefällt mir
1
Einbettungen 0
Keine Einbettungen

Keine Notizen für die Folie

From Site to Inter-site User Engagement

  1. 1. photo  credit  donsolo,  CC  BY-­‐NC-­‐SA  2.0   From  Site  to  Inter-­‐site  User  Engagement   Jane;e  Lehmann   Barcelona,  February  26,  2015     Advisors:  Ricardo  Baeza-­‐Yates   Co-­‐Advisor:  Mounia  Lalmas  
  2. 2. •  User  engagement  is  a  quality  of  the  user  experience  that  emphasizes  the   posiLve  aspects  of  interacLon  with  a  website  –  in  parLcular  the  fact  of  being   capLvated  by  the  website.   •  In-­‐the-­‐moment  engagement                                                                                                                                                                                                     Users  stay  on  a  website  over  a  long  Lme.   •  Long-­‐term  engagement                                                                                                                                                                                                                   Users  come  back  frequently  and                                                                                                                                                                                             over  a  long-­‐term.   IntroducLon   2   User  Engagement   DefiniLon   Successful  websites  are  not  just  used,                                                                           they  are  engaged  with.  
  3. 3. User  Engagement   Measuring   3  IntroducLon   Before  we  can  design  engaging  websites,   it  is  crucial  that  we  are  able  to  measure  engagement.   “If  you  can  measure  it,  you  can  improve  it.”   Sir  William  Thomson   Analysis/Planning   Design  Changes  Measuring  
  4. 4. Main  Research  Goals   4  IntroducLon   Primary  goal    Can  we  define  new  engagement  metrics  that     Measuring      enhance  our  understanding  of  engagement?                 Secondary  goal    Can  we  idenLfy  ways  to  influence  engagement? Analysis/Planning       Analysis/Planning   Design  Changes  Measuring  
  5. 5. IntroducLon   5   Analysis/ Planning   Design  Changes  Measuring  Online  mulLtasking   Inter-­‐site   engagement   Site  engagement   Effect  of  providing   off-­‐site  content   Effect  of  hyperlinks  
  6. 6. IntroducLon   6   Analysis/ Planning   Design  Changes  Measuring   Site  engagement  
  7. 7. Measuring  Engagement   InteracLon  data   7  Site  engagement   Data   Browsing  events  provided  by  Yahoo  toolbar  (client-­‐side).   Engagement   Analysing  the  data  using  online  behaviour  metrics.       Online  session:   Visit  on  Yahoo  News  
  8. 8. Site  engagement   8   Measuring  Engagement   Online  behaviour  metrics   K.  Rodden,  H.  Hutchinson,  X.  Fu.  Measuring  the  user  experience  on  a  large  scale:  User-­‐centered  metrics  for  web  applicaHons.  CHI,  2010.   E.  Peterson,  J.  Carrabis.  Measuring  the  immeasurable:  Visitor  engagement.  Web  AnalyHcs  DemysHfied,  2008.   B.  Haven,  S.  ViWal.  Measuring  engagement.  Forrester  Research,  2008.   B.  Weischedel  and  E.  Huizingh.  Website  opHmizaHon  with  web  metrics:  A  case  study.  Conference  on  Electronic  commerce,  2006.  
  9. 9. Site  engagement   9   Measuring  Engagement   Online  behaviour  metrics   Popularity   #Users   Number  of  users.   #Visits   Number  of  visits.   #Clicks   Number  of  clicks.   AcCvity  (within  a  visit)                                                                                In-­‐the-­‐moment  engagement   PageViews   Avg.  number  of  page  views  per  visit.   DwellTime   Avg.  Lme  on  site  per  visit.   Loyalty  (across  visits)                                                                                                        Long-­‐term  engagement   ReturnRate   Number  of  Lmes  a  user  visited  the  site.   AcLveDays   Number  of  days  a  user  visited  the  site.  
  10. 10. Site  engagement   10   Measuring  Engagement   Differences  in  engagement   ComScore,  Alexa,   GoogleAnalyHcs,…   Shopping   Users  do  not  come   frequently,  but   stay  long   Games   Not  many  users,   but  they  stay  long   News   Users  come   frequently  and     stay  long  
  11. 11. Measuring  Engagement   Problem   11  Site  engagement   Isolated  view:  The  metrics  focus   on  engagement  with  a  single  site.   RelaLonships  to  other  sites  are   not  considered.  
  12. 12. IntroducLon   12   Analysis/ Planning   Design  Changes  Measuring  Online  mulLtasking   Site  engagement  
  13. 13. Online  mulLtasking   13   MoCvaCon   In-­‐the-­‐moment  engagement   ComScore,  Alexa,   GoogleAnalyHcs,…   What  web  analyCcs  think  we  do…   1  visit  with  4  page  views.  
  14. 14. Online  mulLtasking   14   MoCvaCon   In-­‐the-­‐moment  engagement   ComScore,  Alexa,   GoogleAnalyHcs,…   …  and  what  we  really  do:   3  visit  with  on  average  1.3  page  views.  
  15. 15. Online  mulLtasking   15   MoCvaCon   Online  mulLtasking.   Problem     •  Engagement  metrics  do  not  capture  such  behaviour.     •  Measuring  acLvity  on  a  site  can  lead  to  incorrect  conclusions.     Online  mulCtasking   Users  visit  several  sites  and  switch  between  them   during  an  online  session,  to  perform  related  or  totally   unrelated  tasks.  
  16. 16. Research  QuesCon   16  Online  mulLtasking   How  can  we  measure  engagement  by   accounLng  for  user  mulLtasking  behaviour?   Analysis/Planning   Design  Changes  Measuring  
  17. 17. Extent  of  mulCtasking   •  10.2  disLnct  sites,  2  visits  per  site.     Absence  Cme   •  50%  of  sites  are  revisited  aker  <  1min.    InterrupHon  of  a  task     •  There  are  revisits  aker  long  breaks.    Performing  a  new  task   Online  mulLtasking   17   Online  MulCtasking   CharacterisLcs   0.00 0.25 0.50 0.75 1.00 10 2 10 1 10 0 10 1 10 2 Cumulativeprobability Absence time [min] news (finance) news (tech) social media mail 2.09 1.76 2.28 2.09 #Visits Absence time [min] 3.85 3.95 4.47 6.86 Absence time: Time between two visits
  18. 18. AcCvity  paPerns     •  Four  types:  Decreasing,  increasing,  constant,  complex.   •  Successive  visits  can  belong  together  (i.e.  to  the  same  task).   •  Complex  cases  refer  to  no  specific  pa;ern  or  repeated  pa;ern.     Online  mulLtasking   18   Online  MulCtasking   CharacterisLcs   1 2 3 4 ith visit on site 1 2 3 4 ith visit on site 1 2 3 4 ith visit on site 1 2 3 4 ith visit on site Proportionoftotal dwelltimeonsite 0.23 0.28 0.33 p-value = 0.09 m = -0.01 p-value = 0.07 m = -0.02 p-value = 0.79 m = 0.00 news (finance) sitesmail sites social media sites news (tech) sites decreasing attention increasing attention constant attention complex attention
  19. 19. Online  mulLtasking   19   Measuring  Engagement   Online  mulLtasking  metrics   Extent  of  mulCtasking   SessSites   Total  number  of  sites  accessed  (#tasks).   SessVisits   Number  of  visits  to  site  (site  switching).   Absence  Cme   CumAct   Aggregates  the  dwell  Lmes  of  the  visits  with   accounLng  for  the  Lme  between  the  visits.   AcCvity  paPern   A;Shik   A;Range   Describe  the  four  cases  of  a;enLon  shiks.  
  20. 20. 20   CASE  STUDY:   MulCtasking  PaPerns     •  ObjecCve:  Analyse  mulLtasking  acLvity  on  sites;   idenLfy  mulLtasking  pa;erns  (clustering).   •  Metrics:  Site  DwellTime,  MulLtasking  metrics.   •  Data:  July  2012,  2.5M  users,  760  sites  (shopping,   news,  search,  etc.).  
  21. 21. 21   Case  Study:  MulCtasking  PaPerns   Results   No  mulCtasking   MulCtasking   Quick   Focused   Rapid   ConCnuous   Recurring   Checking   weather   Reading   mails   Following  link  to   off-­‐site  content   Purchasing   an  item   Performing   search   Site   DwellTime   -­‐-­‐   ++   ++   ++   -­‐-­‐   Extent  of   mulCtasking   -­‐-­‐   -­‐-­‐   ++   ++   ++   Absence   Cme   -­‐-­‐   ++   ++   ImplicaCons   Provide   interesHng  off-­‐ site  content   Shopping   takes  more  than   one  visit   Support  user   by  finishing   tasks  quickly   Online  mulLtasking   -- low value ++ high value
  22. 22. 22   Case  Study:  MulCtasking  PaPerns   Results   No  mulCtasking   MulCtasking   Quick   Focused   Rapid   ConCnuous   Recurring   Checking   weather   Reading   mails   Following  link  to   off-­‐site  content   Purchasing   an  item   Performing   search   Site   DwellTime   -­‐-­‐   ++   ++   ++   -­‐-­‐   Extent  of   mulCtasking   -­‐-­‐   -­‐-­‐   ++   ++   ++   Absence   Cme   -­‐-­‐   ++   ++   AcCvity   paPern   Online  mulLtasking   De In CmCn 60% 0% De In CmCn 60% 0% De In CmCn 60% 0% Activity pattern: De – Decreasing In – Increasing Cn – Constant Cm - Complex -- low value ++ high value
  23. 23. 23   CASE  STUDY:   Wikipedia  (on-­‐site  mulCtasking)     •  ObjecCve:  Analyse  reading  acLvity  on  Wikipedia   arLcles;  idenLfy  reading  pa;erns  (clustering).   •  Metrics:  ArLcle  DwellTime,  #ArLcles  in  session,   #Views  to  focal  arLcle.   •  Data:  Sep  2011  –  Sep  2012,  500K  users,                             10K  biography  arLcles.  
  24. 24. 24   Case  Study:  Wikipedia   Approach   Online  mulLtasking   Users’  reading  behaviour  on  an  Wikipedia  arCcle   ArLcle  DwellTime        How  much  Lme  do  users  spend  on  an  arLcle?   #ArLcles  in  session      Do  users  view  also  other  arLcles  during  an                  online  session?   #Views  on  focal  arLcle    How  oken  do  users  view  the  arLcle?  
  25. 25. 25   Case  Study:  Wikipedia   Results   No  mulCtasking   MulCtasking   Focus   ExploraCon   Passing   Focus  is  on   focal  arHcle   Exploring  topic   around  the  focal  arHcle   Exploring  topic  and  pass   through  the  focal  arHcle   ArCcle   DwellTime   ++   -­‐-­‐   #ArCcles  in   session   -­‐-­‐   ++   ++   #Views  to             focal  arCcle   ++   -­‐-­‐   ImplicaCons   Content  quality   is  important   Links  to   addiHonal  content   are  important   ArHcles  might   need  to  be  extended   Online  mulLtasking  
  26. 26.   On-­‐site  mulCtasking     •  MulLtasking  between  news  arLcles  of  a  provider.   •  MulLtasking  between  different  tasks  on  a  social  media   site  (e.g.  sharing,  chapng,  updaLng  profile).   •  …     Inter-­‐site  mulCtasking     •  MulLtasking  when  purchasing  items  online  (comparing   offers,  product  reviews,  search,  etc.)   •  …     Online  mulLtasking   26   Further  Use  Cases  
  27. 27. Take  Aways   •  AccounLng  for  mulLtasking  leads  to  a   be;er  understanding  on  how  users   engage  with  sites.     •  Leaving  a  site  does  not  necessarily   entail  less  engagement,  as  users  oken   return  to  the  site  later  on.       Publications J. Lehmann, M. Lalmas, G. Dupret, and R. Baeza-Yates. Online multitasking and user engagement. CIKM 2013. J. Lehmann, C. Müller-Birn, D. Laniado, M. Lalmas, and A. Kaltenbrunner. Reader preferences and behavior on Wikipedia. HT 2014, Ted Nelson Newcomer Paper Award. J. Lehmann, C. Müller-Birn, D. Laniado, M. Lalmas, and A. Kaltenbrunner. What and how users read: Transforming reading behavior into valuable feedback for the Wikipedia community. Wikimania 2014. Online  mulLtasking   27  
  28. 28. IntroducLon   28   Analysis/ Planning   Design  Changes  Measuring  Online  mulLtasking   Inter-­‐site   engagement   Site  engagement  
  29. 29. Inter-­‐site  engagement   29   MoCvaCon   Large  online  service  providers   ComScore,  Alexa,   GoogleAnalyHcs,…   Engagement   Popularity:  #Users,  #Visits,  …   AcLvity:  DwellTime,  PageViews,  …   Loyalty:  ReturnRate,  AcLveDays,  …  
  30. 30. Inter-­‐site  engagement   30   MoCvaCon   Large  online  service  providers   frontpage   tv   sports   shopping   autos   search   daLng   jobs   news   shine   groups   maps   local   health   answer   weather   games   mail   omg   homes   travel   flickr   finance   Large  online  service     providers     (AOL,  Google,  Yahoo,  etc.)     have  not  only  one  site,     but  many  sites.   tumblr  
  31. 31. Inter-­‐site  engagement   31   MoCvaCon   Large  online  service  providers   frontpage   tv   sports   shopping   autos   search   daLng   jobs   news   shine   groups   maps   local   health   answer   weather   games   mail   omg   homes   travel   flickr   finance   Providers  want   that  users  engage  with   many  of  their  sites.   tumblr  
  32. 32. Inter-­‐site  engagement   32   MoCvaCon   Online  mulLtasking   Problem     •  Engagement  metrics  do  not  measure  engagement  across  sites.     •  How  to  adapt  them  is  not  obvious.     Inter-­‐site  engagement   Users  visit  sites  that  belong  to  the   same  network  of  sites.  
  33. 33. Research  QuesCon   33  Inter-­‐site  engagement   How  can  we  measure   engagement  by  also  considering  the   relaLonships  between  sites?   Analysis/Planning   Design  Changes  Measuring  
  34. 34. Inter-­‐site  engagement   34   Traffic  Networks   Modelling   We  model  sites  (nodes)  and  user  traffic   (edges)  between  them  as  a  network.     Provider  network  G=(N,  E,  λ)    N:    Sites    E:    User  traffic    λ(e):  Traffic  volume  (#Clicks)       4  clicks   2  clicks   50  clicks  10  clicks  
  35. 35. Inter-­‐site  engagement   35   Measuring  Engagement   Inter-­‐site  engagement  metrics:  Network-­‐level   Traffic  distribuCon   Flow   Extent  to  which  users  navigate   between  sites.   Density1   Diversity  of  inter-­‐site  engagement.     Reciprocity2   Homogeneity  of  traffic  between  sites.     External  traffic   EntryDisparity   Variability  of  in-­‐going  traffic  to  the   network.     ExitDisparity   Variability  of  out-­‐going  traffic  from   the  network.     [1]  S.  Wasserman.  Social  network  analysis:  Methods  and  applicaHons,  1994.   [2]  T.  SquarHni,  F.  Picciolo,  F.  RuzzenenH,  and  D.  Garlaschelli.  Reciprocity  of  weighted  networks.  Nature:  ScienHfic  reports,  2013.  
  36. 36. Inter-­‐site  engagement   36   Measuring  Engagement   Inter-­‐site  engagement  metrics:  Node-­‐level   Traffic  distribuCon   PageRank1   Probability  that  a  user  will  visit  the   site.   Downstream   Probability  that  a  user  will  conLnue   browsing  to  other  sites.   External  traffic   EntryProb   Probability  that  a  user  enters  the   network  in  this  site.   ExitProb   Probability  that  a  user  leaves  the   network  in  this  site.     [1]  L.  Page,  S.  Brin,  R.  Motwani,  T.  Winograd.  The  pagerank  citaHon  ranking:  Bringing  order  to  the  web.  Technical  report,  Stanford  InfoLab,  1999.  
  37. 37. 37   CASE  STUDY:   Yahoo  Provider  Networks     •  ObjecCve:  Compare  networks;  characterise  the  sites   in  a  network.   •  Metrics:  Network  DwellTime,  Site  DwellTime,  Inter-­‐ site  engagement  metrics.   •  Data:  February  2014,  3.2M  clicks/network,                                   4  country-­‐based  networks,  31  sites  per  network.  
  38. 38. 38   Case  Study:  Yahoo   Comparing  provider  networks   Network  1   Network  2   Network  3   Network  4   High   engaging   Users  engage  quickly   with  many  sites   Users  engage  to  a   subset  of  sites   Low   engaging   Network   DwellTime   ++   -­‐-­‐   ++   -­‐-­‐   Traffic   DistribuCon   ++   ++   Flow  ++   Density  -­‐-­‐   -­‐-­‐   Entry   Disparity   ++   -­‐-­‐   ++   ImplicaCons   The  network   is  performing   well.   This  should   be  looked  into.   MoHvate   users  to  visit   other  sites.   This  should   be  looked  into.   Inter-­‐site  engagement   -- low value ++ high value
  39. 39. 39   Case  Study:  Yahoo   Sites  within  a  provider  network   Traffic  Hub   Supporter   Focused   Engagement   Shared   Engagement   Search,  front  pages   Support,  services   Leisure,  support   News,  leisure   Site   DwellTime   -­‐-­‐   -­‐-­‐   ++   ++   Traffic   DistribuCon   ++   -­‐-­‐   -­‐-­‐   ++   Entry   Probability   ++   -­‐-­‐   ++   -­‐-­‐   ImplicaCons   The  sites   forward  traffic  to   other  sites.   Users  visit  sites   for  specific  needs   and  support.   MoHvate   users  to  visit   other  sites.   The  sites   are  performing   well.   Inter-­‐site  engagement   -- low value ++ high value
  40. 40.   Comparing  networks     •  Device,  Lme,  upstream  traffic,  user.   •  SimulaLons  (effect  of  adding/removing  sites).   •  …     Network  types     •  Network  of  pages  (e.g.  compare  language-­‐based   Wikipedia  networks)   •  Network  of  sites  from  different  providers  (e.g.  shopping   sites,  news  providers)   •  …     Inter-­‐site  engagement   40   Further  Use  Cases  
  41. 41. Take  Aways   •  Inter-­‐site  engagement  allows  for  a   more  comprehensive  look  at  user   engagement  by  also  considering  the   relaLonships  between  sites.       •  Deeply  engaged  users  do  not  only   engage  with  one  site,  but  with  many   sites  in  a  network.       Publications J. Lehmann, M. Lalmas, and R. Baeza- Yates. Measuring Inter-Site Engagement. Handbook of Statistics, Elsevier, 2015. To appear. J. Lehmann, M. Lalmas, R. Baeza-Yates, and E. Yom-Tov. Networked User Engagement. ACM Workshop on User engagement optimization at CIKM, 2013. J. Lehmann, M. Lalmas, and R. Baeza- Yates. Temporal Variations in Networked User Engagement. TNETS Satellite at ECCS, 2013. Some of the metrics were employed to characterise online news reading across news sites: J. Lehmann, C. Castillo, M. Lalmas, and R. Baeza-Yates. Story-Focused Reading in Online News. Submitted for publication. Inter-­‐site  engagement   41  
  42. 42. IntroducLon   42   Analysis/ Planning   Design  Changes  Measuring  Online  mulLtasking   Inter-­‐site   engagement   Site  engagement   Effect  of  providing   off-­‐site  content  
  43. 43. 43   CASE  STUDY:   Online  News     •  Hypothesis:  It  may  be  beneficial  (long-­‐term)  to   enLce  users  to  leave  a  site  by  offering  interesLng   off-­‐site  content.   •  Data:  October  2013,  57K  users,  50  news  sites,   26K  news  arLcles.  
  44. 44. Types  of  reading  sessions     No  click        Did  not  follow  a  related                                                              link.     Off-­‐site  click      Followed  a  related  link  to              content  on  another  site.     Effect  on  engagement     Short-­‐term  Dwell  Lme  per  reading          session.     Long-­‐term  Probability  that  user  starts          next  reading  session  within          the  next  12h.       44   Case  Study:  Online  News   Related  off-­‐site  content   Approach   Effect  of  providing  off-­‐site  content  
  45. 45. Providing  links  to  related  off-­‐site  content  has  a   no  short-­‐term  effect,  but  a  posiCve  long-­‐term  effect.     45   Case  Study:  Online  News   Results   Effect  of  providing  off-­‐site  content   News provider Dwelltimepersession News provider p(absence12h) No Click Off-site click
  46. 46. IntroducLon   46   Analysis/ Planning   Design  Changes  Measuring  Online  mulLtasking   Inter-­‐site   engagement   Site  engagement   Effect  of  providing   off-­‐site  content   Effect  of  hyperlinks  
  47. 47. 47   CASE  STUDY:   Yahoo  Provider  Network     •  Hypothesis:  We  can  use  hyperlinks  to  influence   inter-­‐site  engagement  in  a  provider  network.   •  Data:  February  2014,  235M  clicks,  Yahoo  US   network,  73  sites.  
  48. 48.   Hyperlink  vs.  traffic  network     On-­‐site    Links/Traffic  to  pages          within  the  same  site.     Inter-­‐site  Links/Traffic  to  pages  to          other  sites  in  the          network.     External    Links/Traffic  to          somewhere          else  on  the  Web.   48   Case  Study:  Yahoo   Approach   frontpage   sports   search   news   shine   groups   answer   weather   mail   omg   homes   flickr   Effect  of  hyperlinks  
  49. 49. Hyperlinks  can  be  used  to  influence  site   and  inter-­‐site  engagement  in  a  provider  network.   However,  both  types  of  engagement  influence  each  other.     49   Case  Study:  Yahoo   Results   Effect  of  hyperlinks   Traffic On-site Inter-site External Hyperlinks On-site Inter-site External 0.54 -0.40 - -0.45 0.50 - -0.38 - 0.39
  50. 50. IntroducLon   50   Analysis/ Planning   Design  Changes  Measuring  Online  mulLtasking   Inter-­‐site   engagement   Site  engagement   Effect  of  providing   off-­‐site  content   Effect  of  hyperlinks  
  51. 51. Two  new  perspecHves  for  measuring   engagement  which  consider  the   relaLonships  between  sites.     Online  mulCtasking   Accounts  for  user  mulLtasking   behaviour.     Inter-­‐site  engagement   Accounts  for  the  traffic  between  sites.   ContribuLons  and  future  work   51   Main  ContribuCons   Measuring  engagement   Analysis/ Planning   Design   Changes   Measuring  
  52. 52. AccounLng  for  the  new  perspecLves   when  influencing  engagement.     Online  news   Providing  related  off-­‐site  content   influences  long-­‐term  engagement.     Provider  network   Hyperlinks  affect  site  and  inter-­‐site   engagement,  but  both  influence   each  other.     ContribuLons  and  future  work   52   Main  ContribuCons   Analysis/Planning   Analysis/ Planning   Design   Changes   Measuring  
  53. 53. Wikipedia   Providing  informaLon  about  readers’   engagement  to  the  editor  community.   Yahoo   Using  inter-­‐site  engagement  metrics  to   make  informed  decisions  about  design   changes  (hyperlinks).   Spiegel  Online   Measuring  and  improving  engagement   by  providing  interesLng  off-­‐site   content.   ContribuLons  and  future  work   53   What  next?   Ongoing  and  future  work   Analysis/ Planning   Design   Changes   Measuring  
  54. 54. photo  credit  donsolo,  CC  BY-­‐NC-­‐SA  2.0   Thank  you!     Jane;e  Lehmann     Barcelona,  February  26,  2015   lehmannj@acm.org   Acknowledgements   Ricardo  Baeza-­‐Yates   Mounia  Lalmas   Claudia  Müller-­‐Birn   Carlos  CasLllo   David  Laniado   Andreas  Kaltenbrunner     Elad  Yom-­‐Tov   Georges  Dupret   Guy  Shaked   Fabrizio  Silvestri   Gabriele  Tolomei     Ethan  Zuckerman     John  Agapiou   Andy  Haines   Diego  Sáez-­‐Trumper   Hemant  Purohit   Noora  Al  Emadi   Mohammed  El-­‐Haddad   Nasir  Khan    
  55. 55. •  Mounia Lalmas and Janette Lehmann. “Models of User Engagement”. In H. L. O’Brien and M. Lalmas (Eds.), Why Engagement Matters: Cross-disciplinary Perspectives and Innovations on User Engagement with Digital Media. Springer, 2015, in progress. •  Janette Lehmann, Mounia Lalmas, Elad Yom-Tov, and Georges Dupret. “Models of user engagement.” International Conference on User Modeling, Adaptation, and Personalization (UMAP 2012), pp. 164-175, Montreal, Canada, July, 2012. •  Janette Lehmann, Mounia Lalmas, Georges Dupret, and Ricardo Baeza-Yates. “Online multitasking and user engagement.” ACM International Conference on Information and Knowledge Management (CIKM 2013), pp. 519-528, San Francisco, United States, October, 2013. •  Janette Lehmann, Mounia Lalmas, and Ricardo Baeza-Yates. “Measuring Inter-Site Engagement.”. In V. Govindaraju, V. V. Raghavan, and C. R. Rao (Eds.), Handbook of Statistics, Elsevier, 2015. •  Janette Lehmann, Mounia Lalmas, Ricardo Baeza-Yates, and Elad Yom-Tov. “Networked User Engagement.”, ACM Workshop on User engagement optimization at CIKM, pp. 7-10, San Francisco, United States, October, 2013. •  Janette Lehmann, Mounia Lalmas, and Ricardo Baeza-Yates. “Temporal Variations in Networked User Engagement.”, TNETS Satellite at European Conference on Complex Systems (ECCS), Barcelona, Spain, September, 2013. •  Mounia Lalmas, Janette Lehmann, Guy Shaked, Fabrizio Silvestri, and Gabriele Tolomei. “Measuring Post-click User Experience with Mobile Native Advertising on Streams.”, submitted for publication. •  Janette Lehmann, Claudia Müller-Birn, David Laniado, Mounia Lalmas, and Andreas Kaltenbrunner. “Reader preferences and behavior on Wikipedia.”, ACM International Conference on Hypertext and Social Media (HT 2014), pp. 88-97, Santiago, Chile, September, 2014, Ted Nelson Newcomer Paper Award. •  Janette Lehmann, Claudia Müller-Birn, David Laniado, Mounia Lalmas, and Andreas Kaltenbrunner. “What and how users read: Transforming reading behavior into valuable feedback for the Wikipedia community.”, Presentation at Wikimania, London, UK, August, 2014. •  Janette Lehmann, Carlos Castillo, Mounia Lalmas, and Ricardo Baeza-Yates. “Story-Focused Reading in Online News.”, submitted for publication. •  Janette Lehmann, Carlos Castillo, Mounia Lalmas, and Ethan Zuckerman. “Transient News Crowds in Social Media.” International AAAI Conference on Weblogs and Social Media (ICWSM 2013), Boston, USA, July, 2013. •  Janette Lehmann, Carlos Castillo, Mounia Lalmas, and Ethan Zuckerman. “Finding News Curators in Twitter.” ACM International Conference on World Wide Web Companion (WWW 2013 Companion), 863-870, Rio de Janeiro, Brazil, May, 2013. 55   PublicaCons  
  56. 56. User engagement •  Mounia Lalmas, Heather L O’Brien, and Elad Yom-Tov. Measuring user engagement. Synthesis Lectures on Sample Series #1. Morgan and cLaypool publishers, 2014. •  Heather L O’Brien and Elaine G Toms. What is user engagement? a conceptual framework for defining user engagement with technology. American Society for Information Science and Technology (ASIS&T), 59(6):938–955, 2008. •  Simon Attfield, Gabriella Kazai, Mounia Lalmas, and Benjamin Piwowarski. Towards a science of user engagement (position paper). In Proc. Workshop on User Modelling for Web Applications, WSDM, 2011. •  Kerry Rodden, Hilary Hutchinson, and Xin Fu. Measuring the user experience on a large scale: user-centered metrics for web applications. In Proc. Conference on Human Factors in Computing Systems, CHI, pages 2395–2398. ACM, 2010. Online behaviour metrics •  Brian Haven and Suresh Vittal. Measuring engagement. Forrester Research, 2008. •  Eric T Peterson and Joseph Carrabis. Measuring the immeasurable: Visitor engagement. Web Analytics Demystified, 2008. •  Kerry Rodden, Hilary Hutchinson, and Xin Fu. Measuring the user experience on a large scale: user-centered metrics for web applications. In Proc. Conference on Human Factors in Computing Systems, CHI, pages 2395–2398. ACM, 2010. •  Georges Dupret and Mounia Lalmas. Absence time and user engagement: evaluating ranking functions. In Proc. Conference on Web Search and Data Mining, WSDM, pages 173–182. ACM, 2013. •  Randolph E Bucklin and Catarina Sismeiro. A model of web site browsing behavior estimated on clickstream data. Journal of Marketing Research, 40(3):249–267, 2003. •  Birgit Weischedel and Eelko KRE Huizingh. Website optimization with web metrics: a case study. In Proc. Conference on Electronic commerce: The new e-commerce: innovations for conquering current barriers, obstacles and limitations to conducting successful business on the internet, pages 463–470. ACM, 2006. •  Peifeng Yin, Ping Luo, Wang-Chien Lee, and Min Wang. Silence is also evidence: interpreting dwell time for recommendation from psychological perspective. In Proc. Conference on Knowledge Discovery and Data Mining, SIGKDD, pages 989–997. ACM, 2013. 56   Selected  References  
  57. 57. Online multitasking •  Qing Wang and Huiyou Chang. Multitasking bar: prototype and evaluation of introducing the task concept into a browser. In Proc. Conference on Human Factors in Computing Systems, SIGCHI, pages 103–112. ACM, 2010. •  Hartmut Obendorf, Harald Weinreich, Eelco Herder, and Matthias Mayer. Web page revisitation revisited: implications of a long-term click-stream study of browser usage. In Proc. Conference on Human Factors in Computing Systems, SIGCHI, pages 597–606. ACM, 2007. •  Jeff Huang and Ryen W White. Parallel browsing behavior on the web. In Proc. Conference on Hypertext and Hypermedia, HT, pages 13–18. ACM, 2010. •  Patrick Dubroy and Ravin Balakrishnan. A study of tabbed browsing among mozilla firefox users. In Proc. Conference on Human Factors in Computing Systems, SIGCHI, pages 673–682. ACM, 2010. Inter-site engagement •  Mark EJ Newman. The structure and function of complex networks. SIAM review, 45(2):167–256, 2003. 76, 77, 165 •  Anna Chmiel, Kamila Kowalska, and Janusz A Hołyst. Scaling of human behavior during portal browsing. •  Mark R Meiss, Filippo Menczer, Santo Fortunato, Alessandro Flammini, and Alessandro Vespignani. Ranking web sites with real user traffic. In Proc. Conference on Web Search and Data Mining, WSDM, pages 65–76. ACM, 2008. •  Young-Hoon Park and Peter S Fader. Modeling browsing behavior at multiple websites. Marketing Science, 23(3):280–303, 2004. •  Qiqi Jiang, Chuan-Hoo Tan, and Kwok-Kee Wei. Cross-website navigation behavior and purchase commitment: A pluralistic field research. In Proc. Pacific Asia Conference on Information Systems, PACIS, 2012. •  Kevin Koidl, Owen Conlan, and Vincent Wade. Cross-site personalization: assisting users in addressing information needs that span independently hosted websites. In Proc. Conference on Hypertext and Hypermedia, HT, pages 66–76. ACM, 2014. •  The PEW Research Center. Understanding the participatory news consumer. http://www.pewinternet.org/~/media/Files/ Reports/ 2010/PIP_Understanding_the_Participatory_News_Consumer. pdf, 2010. •  Jason MT Roos. Hyper-Media Search and Consumption. PhD thesis, Duke University, 2012. 57   Selected  References  
  58. 58. Link economy •  Joseph Turow and Lokman Tsui. The hyperlinked society. The University of Michigan Press, 2008. •  Juliette De Maeyer. Hyperlinks and journalism: where do they connect? In Proc. Future of Journalism Conference, 2011. •  Chrysanthos Dellarocas, Zsolt Katona, and William Rand. Media, aggregators, and the link economy: Strategic hyperlink formation in content networks. Management Science, 59(10):2360–2379, 2013. •  Jason MT Roos. Hyper-Media Search and Consumption. PhD thesis, Duke University, 2012. •  Chrysanthos Dellarocas, Zsolt Katona, and William Rand. Media, aggregators, and the link economy: Strategic hyperlink formation in content networks. Management Science, 59(10):2360–2379, 2013. •  Hakan Ceylan, Ioannis Arapakis, Pinar Donmez, and Mounia Lalmas. Automatically embedding newsworthy links to articles. In Proc. Conference on Information and Knowledge Management, CIKM, pages 1502–1506. ACM, 2012. Recommendation •  Richard McCreadie, Craig Macdonald, and Iadh Ounis. News vertical search: when and what to display to users. In Proc. Conference on Research and Development in Information Retrieval, SIGIR, pages 253–262. ACM, 2013. •  Samuel Ieong, Mohammad Mahdian, and Sergei Vassilvitskii. Advertising in a stream. In Proc. Conference on World Wide Web, WWW, pages 29–38. ACM, 2014. •  Eric Sodomka, Sébastien Lahaie, and Dustin Hillard. A predictive model for advertiser value-per-click in sponsored search. In Proc. Conference on Information and Knowledge Management, CIKM, pages 1179–1190. ACM, 2013. •  Narongsak Thongpapanl and Abdul Rehman Ashraf. Enhancing online performance through website content and personalization. Journal of Computer Information Systems, 52(1):3, 2011. •  Jian Wang and Yi Zhang. Utilizing marginal net utility for recommendation in e-commerce. In Proc. Conference on Research and Development in Information Retrieval, SIGIR, pages 1003–1012. ACM, 2011. •  Joshua Porter. Designing for the social web. Peachpit Press, 2010. 58   Selected  References  
  59. 59. ATTACHMENT:   IntroducCon  
  60. 60. •  “In  a  world  full  of  choices  where  the  fleeCng  aPenCon  of   the  user  becomes  a  prime  resource,  it  is  essenLal  that  [...]   providers  do  not  just  design  [websites]  but  that  they   design  engaging  experiences.”  [A}ield].   •  In  addiLon  to  uLlitarian  factors,  such  as  usability  and   usefulness,  we  must  consider  other  factors  of  interacLng   with  websites,  such  as  fun,  fulfillment,  play,  and  user   engagement.   Successful  websites  are  not  just  used,                                                                         they  are  engaged  with.   •  In  order  to  design  engaging  websites,  it  is  crucial  to   understand  what  user  engagement  is  and  how  to   measure  it.   IntroducLon   60   MoCvaCon   Why  is  it  important  to  engage  users?  
  61. 61. Methodology   InteracLon  data,  online  sessions  and  site  visits.   61  IntroducLon   t0 t1 t2 t3 t4 t5 t6 t7 session end session start time Online session Browsing activity on Wikipedia https://ie-mg42.mail.yahoo.com http://en.wikipedia.org/wiki/Freddie˙Mercury http://www.bbc.com/news/uk-29149115 http://www.bbc.com/news/uk-england-nottinghamshire-29643802 http://en.wikipedia.org/wiki/Star Wars http://en.wikipedia.org/wiki/Yoda http://en.wikipedia.org/wiki/Albert˙Einstein https://www.facebook.com/janette.lehmann.5 t0 t1 t2 t3 t4 t5 t6 t7 bc0 bc0 bc0 bc0 bc0 bc0 bc0 bc0 BCookie Timestamp URL - - - http://www.bbc.com/news/uk-29149115 - http://en.wikipedia.org/wiki/Star Wars http://en.wikipedia.org/wiki/Yoda - ReferrerURL Interaction data Page view on Wikipedia Page view on other site
  62. 62. IntroducLon   62   Thesis  structure   Metrics that account for site popularity, activity and loyalty Advertising Chapter 7 Site engagement How users experience ads on desktop and mobile devices? Does ad quality affect the engagement with the publisher? How can we identify high quality ads? Site engagement Chapter 4 Multitasking Chapter 5 Inter-site engagement Chapter 6 Metrics that account for traffic between sites Metrics that account for user multitasking behaviour (III+IV)Applications(II)Fund. Wikipedia Chapter 8 Site engagement and multitasking How users read articles in Wikipedia? Does the activity of editors align with the engagement of readers? How can readers be valuable for editors? Yahoo Chapter 9 Inter-site engagement How users engage with a provider network of sites? Does the hyperlink structure affect site and inter-site engagement? Online news Chapter 10+11 Inter-site engagement How users read stories across news providers? Do hyperlinks to related content influence provider engagement? How can we automatically detect related content? Characterising user engagement Comparing site characteristics and user engagement Applications to impact user engagement
  63. 63. ATTACHMENT:   Site  engagement  
  64. 64. 0-1 1-0.5 0.5 Kendall’s tau with p-value < 0.05 ('-' insignificant correlations) Site  engagement   64   EvaluaCon   CorrelaLons  between  engagement  metrics.   High  correlaCons               within  metric  groups.       Low  correlaCons     between  metric  groups.   [POP]#Users [POP]#Visits [POP]#Clicks [ACT]PageViewsV [ACT]DwellTimeV [LOY]ActiveDays [LOY]ReturnRate #Users [POP] 0.82 0.75 - - 0.43 0.34 #Visits [POP] 0.82 0.85 - - 0.60 0.52 #Clicks [POP] 0.75 0.85 0.16 0.18 0.59 0.51 PageViewsV [ACT] - - 0.16 0.33 - - DwellTimeV [ACT] - - 0.18 0.33 - - ActiveDays [LOY] 0.43 0.60 0.59 - - 0.79 ReturnRate [LOY] 0.34 0.52 0.51 - - 0.79 0.69
  65. 65. Site  engagement   65   PaPerns  of  Site  Engagement   Engagement  depends  on  the  site  at  hand.   Games   Not  many  users,   but  they  stay  long   Search   Users  come   frequently,  but  do   not  stay  long   Social  media   Users  come   frequently  and   stay  long   Shopping   Users  do  not  come   frequently,  but   stay  long   News   Users  come   frequently  and     stay  long   Service   Users  do  not  come   frequently,  but   stay  long  
  66. 66. ATTACHMENT:   MulCtasking  
  67. 67. Online  mulLtasking   67   MoCvaCon   Users  switch  between  sites,  to  perform  related  or  totally  unrelated  tasks.               Switching  between  tasks  (sites)   “…within-­‐session  page  revisits  represent  the  most  common  form  of  revisitaLon,   covering  73,54%  of  all  revisits.”  [Herder]     Performing  tasks  (sites)  in  parallel  using  browser  tabs   “Most  of  our  parLcipants  switched  tabs  more  oken  than  they  used  the  back   bu;on.”  [Dubroy]   [Herder]  E.  Herder.  CharacterizaHons  of  user  web  revisit  behavior.  WWW  Workshop  ABIS,  2005.   [Dubroy]  P.  Dubroy,  R.  Balakrishnan.  A  study  of  tabbed  browsing  among  mozilla  firefox  users.  SIGCHI,  2010.  
  68. 68. Online  mulLtasking   68   Data   Dataset  and  site  categories.   Cat. Subcat. %Sites Description news 22.1% news 5.79% news (soc.) 5.13% society news (sport) 2.63% news (enter.) 2.24% music, movies, tv, etc. news 1.97% news (life) 1.58% health, housing, etc. news (tech) 1.58% technology news (weather) 1.18% service 15.5% service 7.63% translators, banks, etc. mail 3.95% maps 3.03% organisation 0.92% bookmarks, calendar, etc. search 15.3% search 12.63% search (special) 1.58% search for lyrics, jobs, etc. directory 1.05% sharing 9.6% blogging 3.55% knowledge 3.55% collaborative creation and collection of content sharing 2.50% sharing of videos, etc. navi 9.3% front page 6.58% front page (p.) 1.84% personalised front pages sitemap 0.92% leisure 8.7% adult 2.76% games 1.97% social media 1.97% dating 1.05% entertainment 0.92% sites with music, tv, etc.support 8.7% support 1.58% sites that provide products and support for them download 7.11% downloading software shopping 7.9% shopping 4.34% auctions 2.11% comparison 1.45% sites to compare prices of products settings 2.9% login 1.71% site settings 1.18% pr e setting, site personalisation InteracCon  data   •  July  2012   •  2.5M  users   •  785M  page  views   NavigaCon  model   •  We  defined  a  new  navigaLon   model  (see  paper  for  details)     Site  categories   •  760  sites  from  70  countries/ regions   •  11  categories   •  33  subcategories  
  69. 69. Online  mulLtasking   69   MulCtasking  Metrics   CumAct  accounts  for  the  acLvity  between  site  visits.   CumulaCve  acCvity   The  metric  is  defined  as  follows:             InterpretaCon   High  CumAct  à  High  engagement   If  users  return  aker  short  Lme,  they  return  to   conLnue  with  same  task.   If  users  return  aker  longer  Lme,  they  return  to   perform  a  new  task  –  a  sign  of  loyalty.   CumActk = log10 (v1 + ivi k •vi i=2 n ∑ ) Browsing  acLvity  during  the  ith  visit   Browsing  acLvity  between  the  (i-­‐1)th  and  ith  visit   Rescaling  factor  for  ivi       k = 3 vi ivi 1   4  3   10   3   CumAct = log10 (3+13 •4+103 •3) = 3.48 Site  visit  
  70. 70. Online  mulLtasking   70   MulCtasking  Metrics   AWRange  and  AWShik  describe  changes  between  the  visits.   APenCon  shie  and  range   The  metrics  is  defined  as  follows:                     InterpretaCon   AWShik  models  the  shik  of  a;enLon,  and   AWRange  models  the  fluctuaLons   in  the  browsing  acLvity.   AttShiftn = invn − minInvn | maxInvn |− | minInvn | AttRangen = σ (Vn ) µ(Vn ) Variance  in  the  visit  acLvity   Average  of  the  visit  acLvity   Number  of  visits  in  session     ModificaLon  of  the  “Inversion  number”       n = 4 σ µi Inv 0   >0   -­‐1   constant   decreasing   0   constant   complex   +1   constant   increasing   AWenHon  range   AWenHon  shik  
  71. 71. 0-1 1-0.5 0.5 Spearman’s rho with p-value < 0.05 ('-' insignificant correlations) Online  mulLtasking   71   EvaluaCon   CorrelaLons  between  mulLtasking  and  acLvity  metrics.   [MT]SessVisits [MT]SessSites [MT]CumAct [MT]AttShift [MT]AttRange [ACT]DwellTimeS SessSites [MT] 0.42 CumAct [MT] 0.41 - AttShift [MT] 0.09 - - AttRange [MT] - - -0.38 0.27 DwellTimeS [ACT] 0.20 0.24 0.12 0.32 0.08 DwellTimeV [ACT] -0.40 - - 0.14 - 0.50 No  or  only  weak   correlaCons  between                   the  metrics.     All  metrics  convey   different  aspects  about   users’  online  behaviour.  
  72. 72. Online  mulLtasking   72   MulCtasking  PaPerns   Cluster  centers,  site  categories  and  acLvity  pa;erns.   CategoriesMultitasking DwellTimeV CumAct SessVisitsDwellTimeS sitemap site settings news (wheather) download +75% +73% +69% +67% PD 139 sites Quick task Continuous multitasking SessSitesBars from left to right: 111 sites auctions shopping adult dating +79% +71% +71% +62% PD -1.0 1.0 0.0 -1.0 1.0 0.0 PD - Probability difference Activity Activity pattern: De - Decreasing In - Increasing Cn - Constant Cm - Complex De In CmCn De In CmCn 60% 0% 147 sites Recurring task search front page (p.) front page organisation +77% +62% +57% +24% PD -1.0 1.0 0.0 De In CmCn 0.6 0.0 137 sites Focused task news (tech) news (life) support mail +66% +66% +65% +64% PD -1.0 1.0 0.0 -1.0 1.0 0.0 De In CmCn 60% 0% 60% 0% 142 sites Rapid multitasking news (enter.) knowledge comparison service +64% +63% +62% +59% PD -1.0 1.0 0.0 -1.0 1.0 0.0 De In CmCn 60% 0% Single-task-oriented browsing Multitask-oriented browsing
  73. 73. ATTACHMENT:   Inter-­‐site  engagement  
  74. 74. Inter-­‐site  engagement   74   Data   Dataset,  network  and  site  categories.   InteracCon  data   •  August  2013  to  July  2014   •  53M  sessions   Provider  network  G=(N,  E,  λ)      N:    155  Yahoo  sites      from  five  countries      E:    User  traffic      λ(e):  Traffic  volume  (#Clicks)           Site  categories   •  155  sites  from  5  countries   •  5  categories   Cat. %Sites Description 35% 19% 13% 23% 10% news service leisure provider front page mail, calendar, etc. social media, games, etc. account settings, help, etc. front pages, site maps servicefront page news providerleisure
  75. 75. Inter-­‐site  engagement   75   Inter-­‐site  Engagement  Metrics   Flow  accounts  for  the  extent  users  navigate  between  sites.   Traffic  Flow   The  metric  is  defined  as  follows:               InterpretaCon   High  Flow  à  High  inter-­‐site  engagement   Users  navigate  oken  between  the  sites  of  the   network.   Flow = wi, ji, j∑ vii∑ #Clicks  between  node  i  and  j   #Visits  on  node  i   wi, j vi Flow = 30/60 = 0.5 10 5 20 20 20 10 5 1 1 20 20 20 1 1 Flow = 4/60 = 0.07
  76. 76. Inter-­‐site  engagement   76   Inter-­‐site  Engagement  Metrics   Density  describes  the  connecLvity  of  the  network.   Density   We  use  the  density  measure    of  [Wasserman]:             InterpretaCon   High  Density  à  High  inter-­‐site  engagement   Users  navigate  between  many  different  sites   (inter-­‐site  engagement  is  highly  diverse).   [Wasserman]  S.  Wasserman.  Social  network  analysis:  Methods  and  applicaHons,  1994.   Density = # Edges # Possible_ Edges Density = 4/6 = 0.7 Flow = 2/6 = 0.3
  77. 77. Inter-­‐site  engagement   77   Inter-­‐site  Engagement  Metrics   Reciprocity  measures  the  homogeneity  of  traffic  between  two  sites.   Reciprocity   We  use  the  reciprocity  measure  of  [SquarLni]:               InterpretaCon   High  Reciprocity  à  High  inter-­‐site  engagement   Users  navigate  between  two  sites  in  both  direcLons   (inter-­‐site  engagement  is  highly  homogenious).   [SquarHni]  T.  SquarHni,  F.  Picciolo,  F.  RuzzenenH,  and  D.  Garlaschelli.  Reciprocity  of  weighted  networks.  Nature:  ScienHfic  reports,  2013.   #Clicks  between  node  i  and  j  wi, j RP = min[wi, j,wj,i ] i<j∑ wi, ji≠j∑ 1 10 5 20 1 Reciprocity = 15/50 = 0.3 Reciprocity = 2/37 = 0.05 10 10 5 20 5
  78. 78. Inter-­‐site  engagement   78   Inter-­‐site  Engagement  Metrics   Entry/ExitDisp  measures  how  the  traffic  to/from  the  network  is  distributed  over  the  sites.   Entry  disparity  and  exit  disparity   We  use  the  group  degree  measure  of  [Freeman]  and   adapt  it  as  follows:                 InterpretaCon   High  Entry/ExitDisp  à  Low  inter-­‐site  engagement   The  network  is  more  vulnerable  to  outages,  because   only  few  sites  are  used  to  enter  (leave)  the  network.   EntryDisp = (gin max − gin i ) i∑ | N |• gin ii∑ [Freeman]  L.  C  Freeman.  Centrality  in  social  networks  conceptual  clarificaHon.  Social  networks,  1979.   Number  of  visits  that  started  at  node  ni   (user  entered  the  network)   Maximum  value  of  gin   Number  of  nodes  | N | gi in gin max EntryDisp = 20/3 40 = 0.17 20 10 10 40 5 5 EntryDisp = 70/3 50 = 0.47
  79. 79. Inter-­‐site  engagement   79   EvaluaCon:  Network-­‐level   CorrelaLons  between  inter-­‐site  and  network  engagement  metrics.   [IS]Density [IS]Reciprocity [IS]EntryDisparity [IS]ExitDisparity [POP]#Sessions [ACT]DwellTimeS [ACT]#Sites Flow [IS] - 0.15 0.23 0.30 - 0.35 0.65 Density [IS] 0.48 -0.61 -0.60 0.92 -0.45 -0.25 Reciprocity [IS] -0.38 -0.32 0.42 - 0.25 EntryDisparity [IS] 0.84 -0.54 0.33 - ExitDisparity [IS] -0.55 0.38 0.20 0-1 1-0.5 0.5 Spearman’s rho with p-value < 0.01 ('-' insignificant correlations) Density  and  #Sessions   The  more  users  are   visiCng  the  network,  the   more  diverse  is  the  inter-­‐ site  engagement.     Entry-­‐  and  ExitDisparity   Volume  of  in-­‐  and  out-­‐ going  traffic  of  the  nodes   depend  on  each  other.     Flow  and  #Sites   The  more  sites  are  visited   during  a  session,  the   higher  the  flow  of  traffic.  
  80. 80. Inter-­‐site  engagement   80   EvaluaCon:  Node-­‐level   CorrelaLons  between  inter-­‐site  and  site  engagement  metrics.   [IS]Downstream [IS]EntryProb [IS]ExitProb [POP]#Sessions [ACT]DwellTimeS [MT]#Visits [MT]CumAct PageRank [IS] 0.30 -0.08 -0.10 0.85 0.06 0.08 0.31 Downstream [IS] -0.27 -0.22 0.17 0.04 0.02 -0.02 EntryProb [IS] 0.79 0.12 -0.19 0.13 0.35 ExitProb [IS] 0.08 -0.18 0.18 0.32 0-1 1-0.5 0.5 Spearman’s rho with p-value < 0.01 ('-' insignificant correlations) PageRank  and  #Sessions   Popular  sites  in  the   provider  network,  are   also  visited  frequently   when  browsing  through   the  network.     Entry-­‐  and  ExitProb   Nodes  that  are  used  to   enter  the  network  are   also  frequently  used  to   exit  the  network.  
  81. 81. Inter-­‐site  engagement   81   Comparing  Provider  Networks   Country2 Country1 Country3 Country4 Country5 Flow Reciprocity EntryDisparityDensity DwellTimeBars from left to right: -1.0 0.0 1.0 -1.0 0.0 1.0 -1.0 0.0 1.0 -1.0 0.0 1.0 -1.0 0.0 1.0 -1.0 0.0 1.0 -1.0 0.0 1.0 -1.0 0.0 1.0 -1.0 0.0 1.0 -1.0 0.0 1.0 -1.0 0.0 1.0 -1.0 0.0 1.0
  82. 82. Inter-­‐site  engagement   82   PaPerns  of  Inter-­‐site  Engagement  CategoriesEngagement PageRank EntryProb DwellTimeDownstream PD - Probability difference 46 sites Focused eng. front page service provider leisure news +80% +63% -100% -100% -100% PD 23 sites Traffic hub 46 sites Supporter 40 sites Shared eng. CumActBars from left to right: -1.0 1.0 0.0 provider service news front page leisure +31% +19% -2% -10% -100% PD leisure provider service news front page +67% +48% -21% -94% -100% PD news leisure provider front page service +63% -61% -100% -100% -100% PD -1.0 1.0 0.0 -1.0 1.0 0.0 -1.0 1.0 0.0
  83. 83. ATTACHMENT:   NaCve  AdverCsing  
  84. 84. NaLve  AdverLsing   84   Effect  on  User  Engagement   0% 200% 400% 600% short ad clicks long ad clicks adclickdifference short ad clicks long ad clicks clicksperdaydifference PosiLve  experience  has  a  strong  effect  on   users  clicking  on  ads  again,  and  a  small   effect  on  user  engagement  with  the   stream.      
  85. 85. NaLve  AdverLsing   85   Mobile  vs.  Desktop   Ad  post-­‐click  experience  between  mobile   and  desktop  differs.   For  dwell  Lme  we  obtain  rho  =  0.50;  this   value  is  even  smaller  for  bounce  rate  with   rho  =  0.23.     0.00 0.05 0.10 0.15 dwell time difference p(dwelltimedifference) higher on mobilehigher on desktop 0.00 0.05 0.10 0.15 bounce rate difference p(bounceratedifference) higher on mobilehigher on desktop
  86. 86. NaLve  AdverLsing   86   Mobile  OpCmised  Landing  Pages   Dwell  Cme:  The  distribuLon  is  very  similar  for   both  groups.     Bounce  rate:  Decreases  by  6.9%  (median   decreases  by  30.4%)  for  Opt  landing  pages  but   increases  by  13.4%  (median  decreases  by  11.5%)   for  Npt  landing  pages.   not mobile optimized mobile optimized 0.0 0.1 0.2 0.3 dwell time difference p(dwelltimedifference) higher on mobilehigher on desktop mobile opt. not mobile opt. 0.0 0.1 0.2 bounce rate difference p(bounceratedifference) higher on mobilehigher on desktop mobile opt. not mobile opt.
  87. 87. ATTACHMENT:   Wikipedia  
  88. 88. Wikipedia   88   Wikipedia  Research   Literature  review  by  Okoli  et  al.:  The  people’s  encyclopedia  under  the  gaze  of  the  sages:  A   systemaLc  review  of  scholarly  research  on  wikipedia.      
  89. 89. Wikipedia   89   Reading  Preferences   Popularitylow high ArticleLengthshortlong borderline casesII I III IV Jeanne Tsai Douglas Adams Luis Palomino Anne Stears Peter Ehrlich Alec Mango Stephen D. Lovejoy 1st Dalai Lama Dexter Jackson (safety) Katie Green Brittany Borman Anthony Anenih Ronnie Bird Jan Anderson (scientist) Fitch Robertson Sean Bennett For 4.2% (group IV) of the articles editing activity is low, but reading activity is high.!
  90. 90. Wikipedia   90   Reading  PaPerns  Article topic Reading behavior ArticleViewsa SessionArticlesa Popularitya ReadingTimea CA - Percentage in topic 4,826 articles 11,579 behavior vectors sportsperson musician media pers. 28% 26% 23% CA Exploration artist/writer historical fig. polit./businessp. 43% 41% 37% CA 5,278 articles 10,605 behavior vectors Focus 3,876 articles 14,267 behavior vectors historical fig. criminal/victim musican 42% 38% 38% CA Trending 5,684 articles 13,470 behavior vectors media pers. sportsperson musician 27% 27% 19% CA Passing 28K [16K,51K] 11 [5,23] 7.7% 38K [21K,69K] 20 [9,41] 16.9% 26K [15K,45K] 10 [5,21] 10.5% 16K [10K,27K] 8 [3,18] 5.1% ArtLen #Edits %HQA #Edits - Number of edits -1.0 0.5 -0.5 0.0 1.0 -1.0 0.5 -0.5 0.0 1.0 -1.0 0.5 -0.5 0.0 1.0 -1.0 0.5 -0.5 0.0 1.0 %HQA - Percentage of high quality articlesArtLen - Article length
  91. 91. Wikipedia   91   Reading  PaPerns  over  Time   Stability   •  30%  of  the  arLcles  are  popular  in  1  month   •  10%  are  popular  over  the  whole  13-­‐months   •  Almost  all  arLcles  have  one  reading  pa;ern   half  of  their  life  Lme   TransiCons   •  TransiLons  are  temporary  –  arLcles  move   temporarily  to  another  cluster   •  High  reciprocity  –  similar  number  of   transiLons  in  both  direcLons   •  “Focus”  cluster  is  isolated  -­‐  ArLcles  in  that   cluster  are  the  most  stable  ones   •  Strong  connecLon  between  the  “Passing”,   “ExploraLon”,  and  “Trending”  clusters  –   many  arLcles  adopt  all  three  pa;erns  
  92. 92. ATTACHMENT:   Yahoo  
  93. 93. 93   Upstream  Traffic   TeleportaCon   Social  media  /  News   Search  /  Ext-­‐Yahoo   Users  engage  (quickly)   to  many  sites.   Users  conHnue  with   same  acHvity  inside   the  provider  network.   Users  visit  site  they  are   interested  in,  perform  a   quick  task,  and  leave.   Network   DwellTime   -­‐-­‐   ++   -­‐-­‐   Traffic   DistribuCon   ++   -­‐-­‐   -­‐-­‐   Entry   Disparity   -­‐-­‐   Yahoo   Users  engage  differently  depending  on   where  they  are  coming  from.  
  94. 94. 94   Network  Effect  PaPern   Yahoo   Sites  change  their  popularity  in  the  same  way.   Ac>vity  (dwell  >me)  on  a  site  depends  more  on  the  site  itself,   but  there  are  some  nega>ve  dependencies.   Pattern examples 41 patterns Simple star-like 6 patterns Complex star-like 1 pattern Cluster-like 3.00 [3.00,4.00] 0.67 [0.00,0.89] 0 [0,0] 8.00 [7.00,18.00] 0.76 [0.56,0.84] 0 [0,0] 52 0.91 0.51 N Recip Trans N - Number of nodes Recip - Reciprocity Trans - Transitivityservicefront page news providerleisure (4) (5) (6)(1) (2) (3)
  95. 95. 95   Hyperlink  Performance   Yahoo   0% 25% 50% 75% 100% Onsitelinks front page providerservice news leisure Intersitelinks front page providerservice news leisure 0% 20% 40% 60% 80% Externallinks front page providerservice news leisure 20% 40% 60% (a) PageRank and downstream. Traffic PageRank Downstream Hyperlinks PageRank 0.54 - Downstream - - (b) On-site, inter-site, and external. Traffic On-site Inter-site External Hyperlinks On-site 0.54 -0.45 -0.38 Inter-site -0.40 0.50 - External - - 0.39
  96. 96. ATTACHMENT:   Online  News  
  97. 97. Online  news   97   Focused  versus  Non-­‐focused  Sessions   Internal   Non-focused sessionsFocused sessions● (b) Duration (d) p(focused session) (a) %Sessions (f) Flow 25 15 5 60% 20% 0.6 0.2 0.2 0.1 2 3 4 5 6 7 7 2 3 4 5 6 7 7 2 3 4 5 6 7 7 #Articles #Articles #Articles ● ● ● ● ● ● ● (c) #Providers 2.5 2.0 1.5 ● ● ● ● ● ● ● ●● ●● ●● ●● ●● ●● ●● (e) EntryDisparity 0.5 0.3 0.1 ● ● ● ● ● ● ● When  users  focus  on  a  news  story,  they  spend  more  >me  reading  the   ar>cles  and  the  inter-­‐site  engagement  between  providers  is  higher.  
  98. 98. Online  news   98   Hyperlink  Performance     Number  of  Inline  Links   •  <10  links  may  be  wasLng  an  opportunity   •  10-­‐29  links  does  not  result  in  more  clicks   •  >29  links  may  harm  the  user  experience       PosiCon  of  Inline  Links   •  30%  at  the  end,  16%  at  the  beginning,  46%   are  distributed  within  the  arLcle  text.   •  Performance  of  links  located  at  the   beginning  of  the  text  is  very  low  (-­‐28%)   •  Best  performance  is  achieved  with  links  at   the  end  of  the  arLcle  text  (+35%)   Link popularity● Link performance Position in article text Linkpopularity [0.0,0.1[ [0.3,0.4[ [0.6,0.7[ [0.9,1.0] 10% 20% 30% -0.2 0.0 0.2 Linkperformance ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● Number of inline links in article Clicksperlink 0.0 0.2 0.4 0.6 [0,2] [9,11] [18,20] [27,29] [36,38] Number of inline links in article Numberofclicks [0,2] [9,11] [18,20] [27,29] [36,38] 2.5 5.0 7.5
  99. 99. Online  news   99   Effect  on  User  Engagement   Internal  Focused     Short-­‐term:  Only  3  (out  of  50)  providers   have  their  corresponding  average  dwell  Lme   lower  for  the  story-­‐focused  provider   sessions.  The  average  increase  in  dwell  Lme   from  non-­‐story-­‐focused  to  story-­‐focused   provider  sessions  is  50%.       Long-­‐term:  For  78%  of  the  providers,  we   find  that  there  are  more  users  that  return   earlier  aker  they  have  a  story-­‐focused   provider  session.       Internal   News provider Dwelltimepersession Non-focused Focused Ext-focused News provider p(absence12h) Non-focused Focused Ext-focused
  100. 100. Online  news   100   Effect  on  User  Engagement   External  Focused     Short-­‐term:  We  do  not  observe  an  effect  on   the  dwell  Lme  (neither  posiLve  nor   negaLve).  The  average  increase  is  only  5.5%,   and  based  on  the  K-­‐S  test  we  cannot  confirm   that  the  distribuLons  are  different  (p-­‐ value=0.36).       Long-­‐term:  For  70%  of  these  news  sites,  the   probability  that  users  return  within  the   following  12  hours  increases  (the  average   increase  is  76%).         External   News provider Dwelltimepersession Non-focused Focused Ext-focused News provider p(absence12h) Non-focused Focused Ext-focused
  101. 101. Online  news   101   Discovering  Story-­‐related  Content  in  TwiPer  

×