Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
The Songs of Our Past
1. THE SONGS
OF OUR PAST
DOMINIKUS BAUR
WORKING WITH
LISTENING HISTORIES UNIVERSITY OF
MUNICH (LMU)
GERMANY
Hi,
I’m
Dominikus
from
the
University
of
Munich.
I’m
a
fourth
year
Ph.D.
student
and
today
I
will
talk
about
some
of
the
work
that
I’ve
done
so
far.
2. In this talk:
As
you’ve
probably
already
guessed
from
the
Btle,
my
focus
is
on
a
special
type
of
personal
histories,
namely
music
listening
histories.
In
this
talk
I
will
first
describe
what
listening
histories
are
and
what
we
mean
by
that.
Then
I’ll
show
you
some
projects
from
the
area
of
informaBon
visualizaBon
where
we
worked
with
listening
histories
from
single
or
mulBple
people
and
tried
to
make
them
understandable
for
them.
Finally
I
will
give
you
some
ideas
what
else
than
visualizing
we
could
do
with
this
type
of
data.
3. Photos,
be
they
analog
or
digital,
are
a
common
way
to
remember
the
past.
We
all
take
photos
while
on
vacaBon,
having
friends
over
or
for
all
these
other
occasions
and
aIerwards
look
at
them
(or
don’t)
and
think
about
the
past.
But
what
we
do
in
our
lives
is
oIenBmes
so
much
more
than
a
photo
can
capture
4. [click]
unfortunately
all
auditory
informaBon
is
lost
in
the
process.
The
music
we
made,
the
songs
we
heard.
Nowadays,
of
course,
there
is
oIen
a
structural
difference
between
both
acBviBes:
The
Bme
we
spend
acBvely
making
music
is
significantly
smaller
than
the
Bme
we
spend
listening
to
it.
5. But
even
though
we
no
longer
make
the
music,
it
is
more
abundant
than
ever
before
thanks
to
our
mobile
gadgets.
And
so
these
songs
by
people
we
don’t
know
sBll
stand
for
parts
of
our
lives:
7. Or
the
one
that
was
playing
when
you
met
a
special
someone…
8. …
and
of
course
the
songs
we
hear
every
year
for
special
occasions.
9. REMINISCING
So,
an
account
of
all
the
music
we
listened
to,
a
listening
history,
can
serve
for
reminiscing
just
as
well
as
photos.
In
this
regard,
listening
histories
are
a
part
of
the
so-‐called
lifelogging
data.
10. Lifelog
A digital representation of all
aspects of one’s life
Lifelogs
are
digital
representaBons
of
aspects
of
one’s
life.
So,
via
this
definiBon,
every
facebook
status
and
blog
entry
already
stands
as
a
part
of
lifelog
data.
But
the
original
vision
of
lifelogging
consists
of
capturing
really
everything
that
you
experience.
And
the
original
visionaries
went
…
11. …
to
great
lengths
to
reach
that
goal.
So
while
capturing
listening
histories
is
only
a
humble
secBon
of
a
complete
lifelog,
they
can
sBll
bring
many
of
the
benefits.
12. REMINISCING
Sellen, Whittaker: Beyond Total Capture: A
Constructive Critique of Lifelogging,
CACM, May 2010
In
a
recent
paper,
Abigail
Sellen
and
Steve
WhiXaker
idenBfied
some
of
the
benefits
that
lifelogging
data
can
bring
and
summarized
them
as
the
‘5
Rs’.
We’ve
already
seen
reminiscing,
as
re-‐living
the
past
for
emoBonal
reasons.
14. RECOLLECTING
REMINISCING
RETRIEVING
REFLECTING
REMEMBERING Sellen, Whittaker: Beyond Total Capture: A
Constructive Critique of Lifelogging,
CACM, May 2010
RecollecBng
is
the
more
general
(and
less
emoBonal)
case
of
reminiscing
and
can,
for
example,
mean
using
a
listening
history
to
find
a
song
whose
name
I
have
forgoXen.
15. RECOLLECTING
REMINISCING
RETRIEVING
REFLECTING
REMEMBERING Sellen, Whittaker: Beyond Total Capture: A
Constructive Critique of Lifelogging,
CACM, May 2010
Retrieving
is
more
appropriate
for
text-‐
and
other
documents,
but
it
can
also
mean
that
I
can
immediately
listen
to
that
song.
16. RECOLLECTING
REMINISCING
RETRIEVING
REFLECTING
REMEMBERING Sellen, Whittaker: Beyond Total Capture: A
Constructive Critique of Lifelogging,
CACM, May 2010
ReflecBng
describes
the
process
of
thinking
about
your
life
using
the
lifelog.
Say,
I
listened
to
a
fair
share
of
pop
and
rock,
now
it’s
Bme
to
become
serious
and
listen
to
classical
music.
17. RECOLLECTING
REMINISCING
RETRIEVING
REFLECTING
REMEMBERING Sellen, Whittaker: Beyond Total Capture: A
Constructive Critique of Lifelogging,
CACM, May 2010
And
finally,
remembering
intenBons
describes
thinking
about
prospecBve
acBviBes,
such
as
regularly
checking
if
a
band
has
a
new
album
or
is
on
tour.
18. RECOLLECTING
REMINISCING
RETRIEVING
REFLECTING
REMEMBERING Sellen, Whittaker: Beyond Total Capture: A
Constructive Critique of Lifelogging,
CACM, May 2010
So,
these
‘five
Rs’
provide
a
good
overview
of
the
possible
benefits
of
capturing
listening
histories.
19. Let
me
talk
a
liXle
bit
about
what
listening
histories
are
and
where
they
come
from.
20. Listening history
A complete chronological
collection of musical items
…
In
my
understanding
an
ideal
listening
history
describes
all
songs
that
a
person
has
listened
to,
possibly
in
their
lifeBme.
What’s
important
here
is
that
…
21. ...
Each song:
(1) pre-existing piece of music
...
Each
song
is
a
pre-‐exisBng
piece
of
music
that
has
aXributes
such
as
arBst,
Btle,
etc.
22. …
(2) has been heard at least partially
And
second,
each
song
has
been
heard
by
the
owner
of
the
history
at
least
in
parts.
So,
the
quesBon
is,
where
do
we
get
such
data
from?
23. Fortunately,
there’s
a
popular
service
called
‘last.fm’
that’s
been
around
for
a
while
and
does
exactly
that.
Last.fm’s
actual
intenBon
for
capturing
a
person’s
listening
behavior
is
providing
beXer
recommendaBons
for
their
webradio,
but
the
resulBng
listening
histories
are
easily
accessible
through
their
API
which
makes
them
a
perfect
target
for
all
kinds
of
projects.
24. Last.fm’s
tracking
technology
is
called
‘Audioscrobbler’,
which
is
both
a
protocol
and
a
soIware.
Devices
and
media
players
can
either
use
the
protocol
directly
or
rely
on
the
background
audioscrobbler
process
running
on
the
user’s
machine.
And
in
the
end
we
arrive
at
a
chronological
list
of
/all/
the
songs
a
person
has
listened
to...
25. +
=
But
while
this
could
be
used
to
hypotheBcally
capture
the
complete
listening
history
of
a
person
and
works
great
in
theory
[click]
27. Real listening
histories:
- incomplete
- noisy
The
actual
resulBng
listening
histories
are
both
incomplete
and
noisy.
Let
me
just
tell
you
what
I
mean
by
that.
28. Real listening
histories:
- incomplete
- noisy
Gaps
in
a
listening
history
can
come
from
various
places…
29. One
common
source
is
that
the
listener
is
using
non-‐supported
hardware
for
listening
30. Another
that
music
comes
from
other
sources
like
when
shopping
or
being
at
a
friend’s
place.
31. Real listening
histories:
- incomplete
- noisy
Noise,
i.e.,
too
many
songs
are
tracked
is
also
quite
common…
32. The
user
might
leave
the
computer
while
the
music
keeps
on
playing…
33. Or
someone
else
is
using
the
computer
while
the
audioscrobbler
is
sBll
running.
34. > 50%
Another Caveat: The audioscrobbler only tracks a song after the user has listened to at
least half of it. Again, keep in mind that the main incentive for last.fm to track listening
is improving the recommendations of their web radio: If a song is skipped then the
listener probably didn’t like it and it’s uninteresting for recommendations.
35. 30 million users / month
(March 2009)
http://blog.last.fm/2009/03/24/lastfm-radio-announcement
SBll,
despite
these
downsides,
last.fm’s
data
is
preXy
reliable
and
the
service
is
very
popular.
According
to
them,
30
million
people
visit
the
webpage
per
month.
36. In
the
end
we
arrive
at
a
chronological
list
of
songs
and
that’s
all
we
get.
Each
secBon
of
Bme
either
contains
music
or
it
does
not.
So
we
have,
for
example,
no
informaBon
on
the
context
of
the
music
listening
(I’ll
get
back
to
that
aspect
later).
SBll,
to
make
it
easier
to
understand
this
data
we
can
then
start
to
analyze
it
and
e.g.,
to
extract
listening
sessions.
37. Listening
sessions
are
characterized
by
the
gaps
between
the
songs,
so
a
gap
of
e.g.,
half
an
hour
between
two
songs
means
that
the
creator
of
the
history
stopped
listening
and
thus
ended
the
session.
To
make
these
histories
a
bit
more
meaningful
we
can
also
go
beyond
the
single
Bme
dimension…
38. Genre
……
Sub-Genre
Artists
Albums
Songs
…
and
put
the
songs
into
the
musical
hierarchy
of
albums,
arBsts
and
genres.
While
this
classificaBon
is
not
perfect
and
oIenBmes
the
topic
of
heated
debates,
at
least
it’s
widely-‐
known
among
all
music
listeners.
One
more
step
to
overcome
the
downsides
of
a
strict
hierarchy
is
adding
user-‐generated
keywords
into
the
mix…
39. Genre
……
Sub-Genre
Artists
Albums
Songs
Tags
…
that
can
become
a
stand-‐in
for
any
number
of
different
hierarchies
or
classificaBons.
You
will
see
some
of
these
aspects
in
the
prototypes
that
I’m
about
to
show
you.
40. But
back
to
the
actual
benefits
to
the
creators
of
such
listening
histories,
think
of
the
‘5
Rs’
of
reminiscing,
recollecBng
and
so
on.
Here
you
can
see
the
default
view
of
Last.fm
presenBng
this
data:
A
chronological
web-‐based
list
which
is
not
that
helpful
for
any
of
these
tasks.
And
as
you’ve
just
seen,
listening
histories
can
become
quite
complex
once
you
dive
into
their
depths
which
makes
other
forms
of
presentaBon
more
useful.
41. As
a
first
step
towards
understanding
this
data
and
also
making
their
owners
understand
them
I
started
building
visualizaBons
based
on
it.
To
make
maXers
not
overly
complicated,
…
42. …
I
used
only
a
single
listening
history,
i.e.
a
possibly
long
list
of
possibly
repeated
songs.
43. My
first
approach
to
visualizing
this
type
of
data
was
a
node-‐link
diagram:
The
idea
was
that
each
unique
song
would
be
represented
as
a
node
…
44. …
while
each
pair
of
consecuBve
songs
would
form
an
edge
in
the
diagram.
And
while
this
concept
was
easy
to
understand,
the
result
wasn’t
–
necessarily.
And
you
might
also
understand
why
I
Btled
this
visualizaBon
‘ Tangle’:
45. Here
be
a
chaoBc
screenshot
of
tangle
Tangle
While
it
certainly
looks
chaoBc,
there
are
sBll
several
aspects
that
you
can
draw
from
it:
46. Here
be
a
chaoBc
screenshot
of
tangle
Tangle
For
one,
the
layout
of
the
nodes
is
force-‐directed,
which
means
that
nodes
with
many
edges
(i.e.
songs
that
appear
repeatedly
within
the
history)
are
drawn
towards
the
center,
…
47. Here
be
a
chaoBc
screenshot
of
tangle
Tangle
…
while
less
popular
songs
and
one-‐hit-‐wonders
are
on
the
outskirts.
48. Here
be
a
chaoBc
screenshot
of
tangle
Tangle
An
addiBonal
encoding
is
the
thickness
of
the
connecBng
arrows
that
represents
the
number
of
Bmes
this
two-‐song-‐sequence
was
played
which
shows
albums
and
pre-‐defined
playlists.
49. VIDEO
Tangle
[VIDEO]
And
finally,
Tangle’s
layout
is,
as
I
said,
force-‐directed
which
means
that
the
user
is
able
to
interacBvely
explore
the
visualizaBon.
Zooming
and
panning
is
of
course
possible.
By
hovering
over
a
song
addiBonal
informaBon
is
shown.
And
the
user
can
drag
around
songs
at
will.
50. As
it
was
not
easy
to
learn
much
from
the
Tangle
visualizaBon,
I
wanted
to
put
some
sense
into
it.
Filtering
or
splihng
the
data
seemed
promising,
so
I
focused
on
listening
sessions
this
Bme.
51. The
basic
idea
was
again
a
node-‐link
diagram,
but
this
Bme
songs
could
appear
more
than
once.
52. This
Bme
the
more
important
factors
were
the
Bme
stamps
of
the
songs.
A
pause
of
in
this
case
1
hour
indicates
the
start
of
a
new
listening
session.
53. Strings
By
sorBng
the
sessions
chronologically
we
arrived
at
this
visualizaBon,
called
‘Strings’.
54. Strings
Zooming
out
gives
you
an
overview
of
the
length
of
your
listening
sessions,
shows
outliers
and
Bmes
when
you
didn’t
listen
to
music.
The
verBcal
Bme
line
is
very
important
in
this
regard.
55. Strings
Finally,
you
probably
wondered
about
the
blue-‐ish
arcs:
The
problem
with
Strings
is
that
each
song
can
possibly
appear
several
Bmes
in
the
visualizaBon
as
single
songs
are
no
longer
represented
by
single
nodes.
Therefore,
we
draw
arcs
between
idenBcal
songs
which
makes
it
possible
to
gauge
the
importance
of
one
song
or
see
repeBBve
sequences
(at
the
boXom).
56. ?
So,
what
these
two
examples
had
in
common
that
they
were
both
restricted
visualizaBons
that
(1)
focussed
on
one
aspect
of
the
data
and
(2)
allowed
only
liXle
interacBon.
57. playful
If
you
want
to
put
a
label
on
them
it
would
probably
be
‘playful’
which
means:
They
are
designed
for
one
specific
aspect
of
the
data
which
cannot
be
customized.
They’re
built
for
this
task
only.
But
they
can
sBll
engage
the
user
to
play
around
and
interact
(at
least
a
liXle).
58. playful casual expert
If
you
want
to
put
this
into
an
infovis
perspecBve,
two
other
commonly
used
terms
are
useful:
‘Casual’
describes
visualizaBons
that
are
a
liXle
more
interacBve
and
customizable
but
not
as
complex
as
‘expert’
systems
that
allow
fine-‐grained
customizaBon
but
require
solid
knowledge
in
the
respecBve
area.
59. playful casual expert
For
visualizaBons:
A
type
of
playful
visualizaBon
would
be
Wordle,
engaging
but
with
a
single
purpose.
The
Many
Eyes
project
is
easy
to
use
but
has
much
more
ways
to
display
and
filter
the
data.
Finally,
programming
frameworks
such
as
protovis
or
processing
allow
utmost
flexibility
but
are
difficult
to
get
into
and
master.
60. Interactivity
expert
casual
playful
Complexity
If
we’re
inclined
to
put
these
three
concepts
into
relaBon
to
each
other,
we
can
use
interacBvity
and
complexity.
So,
playful
tools
aren’t
very
flexible,
but
also
not
very
complex.
Expert
tools
however
are
mulB-‐purpose
and
highly
interacBve
but
also
difficult
to
master.
It
depends
on
the
user
populaBon
and
the
task
what
visualizaBon
concept
to
choose.
61. Interactivity
expert
casual
playful
Complexity
In
our
case
with
listening
histories,
we
have
people
who
like
to
listen
to
music
and
are
not
necessarily
infovis-‐experts.
Also,
analyzing
their
listening
behavior
is
something
they
don’t
do
regularly
so
forcing
them
to
learn
something
for
using
a
complex
visualizaBon
will
rather
put
them
off
than
engage
them.
Therefore
I
concentrated
on
the
playful/casual
corner
of
this
design
space.
62. Ok,
so
back
to
the
visualizaBon.
Both
Strings
&
Tangle
were
very
single
purpose
and
liXle
customizable.
For
the
next
project,
I
wanted
to
give
users
more
freedom
in
analyzing
their
listening
histories
but
sBll
keep
the
tool
accessible.
Strings
&
Tangle
were
also
only
informally
evaluated
with
a
few
people
from
our
lab
so
I
wanted
to
see
if
real
people
would
actually
find
something
like
that
useful…
63. LastHistory
...
The
result
was
LastHistory,
a
/casual/
infovis
tool
for
analyzing
and
reminiscing
in
one’s
own
listening
history.
We
made
it
available
on
the
internet.
Several
thousand
people
downloaded
it
and
we
received
lots
of
feedback.
When
designing
LastHistory
we
first
wanted
to
make
sure
that
it
felt
easily
accessible
for
people.
The
visualizaBon
in
its
non-‐interacBve
state
should
already
give
insights
to
the
user,
and
so
gradually
lure
them
into
exploring
the
more
sophisBcated
opBons.
64. LastHistory
So,
the
largest
part
of
the
applicaBon
is
taken
up
with
a
2D
Bmeline:
all
songs
are
represented
as
small
circles
and
mapped
horizontally
to
the
day
and
verBcally
to
the
Bme
of
day
of
their
Bmestamps.
This
way,
users
can
easily
see
daily
rhythms,
66. LastHistory
And
here’s
another
example:
A
user
who
gets
up
at
the
same
Bme
everyday
and
listens
to
music
first
thing
in
the
morning.
67. classical
jazz
funk
hip-hop
electronic
rock
metal
unknown/other
LastHistory
Each
song’s
genre
is
color-‐coded,
so
the
user
gets
an
immediate
overview
over
the
variety
of
songs.
We’re
of
course
restricted
in
the
number
of
colors
we
can
use
to
keep
them
disBnguishable.
69. LastHistory
One-‐dimensional
zooming
by
using
the
mouse’s
zoom
wheel
or
the
slider
in
the
lower
right
corner
allows
them
to
focus
on
certain
secBons
of
the
history.
70. LastHistory
Hovering
over
a
song
shows
a
box
with
user-‐generated
keywords
from
last.fm,
but
more
prominently:
connects
this
song
with
all
other
instances
of
it
throughout
the
history.
So,
users
can
easily
see
when
they
listened
to
this
one
song.
71. LastHistory
Preceding
and
succeeding
repeated
songs
are
also
highlighted,
so
sequences
such
as
albums
or
other
predefined
playlists
are
automaBcally
highlighted.
72. LastHistory
Finally,
in
the
upper
right
corner
of
the
applicaBon,
there’s
a
textbox
for
filtering
where
users
can
enter
freeform
terms.
It’s
possible
to
enter
song
or
album
Btles
or
arBst
names
to
filter
all
other
songs.
73. LastHistory
But
the
filter
box
can
also
be
used
for
temporal
queries
by
entering
dates,
or
periods
of
Bme,
so
users
can,
for
example,
see
all
songs
that
they
listened
to
in
autumn
before
noon.
But
enough
with
the
default
infovis-‐features.
One
interesBng
aspect
of
this
project
was
that
we
could
use
an
addiBonal
data
source
for
gaining
insights:
The
user’s
memories.
74. To
access
these,
we
needed
memory
triggers.
Some
research
in
psychology
has
shown
that
personally
created
things
such
as
photos
can
be
useful
in
this
regard,
so
we
integrated
photos
and
calendar
entries
from
the
user’s
harddisk.
75. Two usage modes:
Analysis Personal
We
split
these
into
two
different
usage
modes
and
called
them
‘analysis’
(everybody
can
do
it)
and
‘personal’
(with
memory
triggers
that
probably
are
only
useful
to
the
owner
of
the
history).
So
in
this
personal
mode
we
have
contextual
informaBon
that
makes
it
easier
to
remember
what
happened
at
what
Bme
and
understanding
the
listening
decisions.
Users
could
simply
switch
between
the
two
modes
with
the
buXon
in
the
upper
leI
corner.
76. Ok,
so
much
for
the
tool.
As
I
said,
we
made
it
available
on
the
internet
and
a
lot
of
people
downloaded
it.
77. Praise on tech blogs
5,000 downloads
243 filled-out questionnaires
Some
numbers:
First
we
got
a
good
amount
of
coverage
on
tech
blogs,
which
led
to
a
certain
popularity.
Right
now,
we
have
about
5,000
downloads.
We
also
included
a
link
to
a
quesBonnaire
that
pops
up
aIer
fiIeen
minutes
of
using
the
tool
and
around
250
people
answered
that
quesBonnaire.
We
kept
that
intenBonally
short,
in
order
not
to
put
off
people
as
a
short
answer
is
beXer
than
no
answer
at
all.
78. About the Personal Mode:
“I like this mode the best, it
should be the default mode!”
“Clicking on a photo gallery and
listening to what I was listening
to at the time was very powerful.”
People
who
had
photos
and
calendar
entries
available
enjoyed
using
the
personal
mode
and
also
features
like
the
possibility
to
create
a
slideshow
of
music
and
photos.
79. About the Analysis Mode:
“I rarely listen to music between
the hours of 9-11 a.m., even on
weekends.”
“I noted the … commuting pattern.”
“Those ruts where you get stuck in
listening to one particular song.”
“I listened to music for 4 straight
days!”
And
in
general,
people
were
also
able
to
find
repeaBng
paXerns
and
liked
how
they
were
able
to
learn
interesBng
aspects
about
themselves.
80. 75% found it easy to use and learn.
Another
thing
we
learned
that
worked
really
well
was
puhng
a
five
minute
video
online
that
explained
how
to
use
the
tool:
There
was
no
online
help
or
something
like
that
available
and
sBll
75%
percent
found
it
easy
to
learn
and
use.
81. Finally,
people
really
liked
to
share
the
results.
And
as
there
was
no
straighporward
way
to
do
it
within
the
applicaBon,
they
resorted
to
taking
screenshots
for
posBng
it
on
flickr
or
their
blogs.
So
a
future
version
should
definitely
take
that
into
account.
82. A
Ok,
so
these
were
three
examples
for
visualizing
single
listening
histories.
But
it
gets
much
more
interesBng
when
we
have
not
one
history…
83. E
D
C
B
A
…
but
many
more.
And
as
music
has
an
intricate
social
funcBon
as
well,
comparing
one’s
taste
in
music
to
friends,
family
and
peers
can
be
an
interesBng
use
case.
84. So
in
the
following,
I
will
give
you
two
examples
for
approaches
to
visualize
these
data.
Again,
I’ll
present
one
playful
and
one
casual
approach.
85. B
A
While
mulBple
histories
can
mean
a
lot
of
histories,
for
a
first
approach,
I
decided
to
focus
on
just
two
histories.
One
will
usually
be
the
user’s
history
and
the
other
one
of
a
friend
or
another
person
that
he
or
she
knows.
86. Ok,
so
what’s
the
best
way
to
do
that:
Aligning
the
songs
to
a
Bme-‐line
is
probably
a
good
idea,
to
allow
comparisons
for
the
number
of
songs,
regularity
of
listening
and
so
on.
But
users
are
especially
in
this
one-‐on-‐one
scenario
interested
in
also
comparing
their
taste:
Are
both
of
them
listening
to
the
same
songs
or
arBsts?
Or
is
there
no
similarity?
87. An
easy
way
to
encode
that
is
using
the
distance
between
the
songs
from
each
history:
The
closer
a
song
gets
to
the
other
history,
the
more
similar
it
is
to
it,
resulBng
in
a
fever
chart
of
relatedness.
88. LoomFM
Here’s
an
example
from
the
resulBng
‘LoomFM’
visualizaBon.
You
have
a
horizontal
Bmeline
and
two
listening
histories
from
user
red
and
purple.
The
closer
one
of
the
small
song
circles
comes
to
the
Bmeline,
the
more
related
it
is
to
the
other
user’s
taste
in
music.
89. LoomFM
Some
more
things:
The
more
consecuBve
songs
share
the
same
genre
or
arBst,
the
larger
the
corresponding
label
gets.
By
doing
this,
important
arBsts
are
visible
even
when
zoomed
out.
Also,
labels
that
both
users
share
at
the
same
point
in
history
move
to
the
center
of
the
Bmeline.
90. LoomFM
AddiBonally,
the
yellow
arcs
connect
idenBcal
songs
–
the
same
principle
as
in
the
‘Strings’
visualizaBon.
Using
this
approach,
you
can
get
a
sense
if
a
person
repeatedly
listens
to
the
same
songs
(as
user
red
in
this
example)
or
only
once.
Also,
songs
that
both
users
share
are
connected…
91. LoomFM
…
as
in
this
example,
where
a
new
album
of
‘ Trail
of
Dead’
was
released
and
both
users
gradually
started
listening
to
it.
(you
can
also
clearly
see
the
sequence
here)
92. VIDEO
LoomFM
Here
you
can
see
a
video
of
LoomFM.
As
always,
zooming
and
panning
are
possible
and
gehng
more
informaBon
by
hovering
over
songs
or
arcs.
93. playful casual expert
LoomFM
consBtutes
an
example
of
a
playful
visualizaBon
for
mulBple
histories.
The
tasks
are
clearly
defined,
interacBon
is
minimal
and
a
lot
of
informaBon
(e.g.
the
similarity
between
songs)
is
implicit
and
predefined…
94. Screenshot
Of
LastLoop
playful casual expert
…
to
overcome
the
restricBons
and
also
to
integrate
more
than
two
histories
we
did
another
project
called
LastLoop
and
aimed
more
for
the
casual
area.
95. The
basic
idea
was
to
have
a
cross
between
LastHistory
and
LoomFM,
to
give
users
the
chance
to
do
these
more
complex
analyses
using
filtering
and
things
like
that
while
also
being
able
to
connect
the
different
listening
histories
and
see
relaBons
between
them.
96. LastLoop
Here’s
the
result
that
we
called
‘LastLoop’.
What
you
can
see
here
are
three
listening
histories
(you
can
have
an
unlimited
number
of
verBcally
stacked
histories),
arranged
to
the
same
Bmeline.
97. LastLoop
We
used
the
2D
Bmeline
metaphor
from
LastHistory
once
more,
so
it’s
possible
to
see
daily
paXerns
across
all
histories.
98. LastLoop
Also,
by
hovering
above
a
song,
all
other
occurences
within
this
one
history
and
the
others
are
highlighted
(re-‐using
the
metaphor
from
the
other
projects).
99. LastLoop
The
user
can
also
select
a
whole
area
and
again,
see
where
else
the
songs
appear.
100. LastLoop
Finally,
to
make
the
informaBon
manageable,
users
can
also
search
for
songs,
arBsts,
albums
and
so
on…
102. VIDEO
LastLoop
And
here’s
the
system
in
acBon:
You
can
pan
and
zoom
either
by
using
the
mouse
or
the
Bme
slider
at
the
boXom
of
the
screen,
select
screen
regions
to
see
other
occurences
of
the
selected
songs
(and
switch
between
all,
songs
from
the
selecBon
or
songs
within
the
selecBon
only).
Aaaaaand
you
can
also
highlight
songs
or
arBsts
…
and
filter
for
certain
genres.
103. http://www.lastloop.de
So,
to
evaluate
the
system
we
followed
the
same
strategy
that
had
already
worked
with
LastHistory.
We
made
the
tool
available
on
the
web
(and
this
Bme
it
was
even
wriXen
in
Java
and
thus
plaporm-‐independent,
while
LastHistory
was
Mac-‐only).
You
could
-‐
and
sBll
can
-‐
run
it
easily
in
your
browser.
104. For
learning
the
applicaBon,
we
provided
another
five-‐minute-‐video
that
explained
the
basics
of
interacBon
and
to
capture
the
users’
findings
we
had
another
short
quesBonnaire
…
105. …
and
we
also
had
‘feedback’
buXon
in
the
upper
leI
of
the
applicaBon
where
users
could
click
on,
provide
what
they
found
and
send
it
directly
back
to
us.
106. 21 filled-out questionnaires
(3 incomplete)
So,
while
we
were
preXy
convinced
that
we
did
everything
right,
the
response
was
less
than
stellar.
AIer
one
month
we
had
21
responses
to
the
quesBonnaire
and
a
few
with
the
direct
feedback
buXon.
107. Insights gained:
“That one user is also listening to
a very infamous band from the 70s”
“When did the other user hear my
favorite song, have there been many
connections lately, …”
What
we
found
was
that
people
learned
about
themselves
and
others,
which
was
the
goal
of
the
visualizaBon
and
we
were
happy
that
it
worked.
But
we
wanted
to
find
out
what
went
wrong…
108. Selecting a song was sketchy
Results were cluttered and unclear
…
and
the
problems
were
mostly
due
to
usability
issues
and
the
general
complexity
of
the
applicaBon.
People
found
it
difficult
to
accurately
select
a
song
as
the
selecBon
was
only
based
on
the
horizontal
posiBon
of
the
cursor
and
not
the
verBcal
(so
it
became
very
hard
to
select
a
specific
song
when
zoomed
out).
Also,
people
liked
how
the
results
looked
but
couldn’t
make
much
sense
of
them.
It
was
oIen
just
too
much
informaBon
in
too
liXle
space,
so
drawing
any
insights
other
than
very
superficial
ones
was
difficult.
109. Screenshot
Of
LastLoop
playful casual expert
So
what
we
learned
was
that
even
when
we
fixed
the
usability
issues,
LastLoop
would
probably
sBll
be
more
of
an
expert-‐
than
a
casual
visualizaBon.
110. Screenshot
Of
LastLoop
playful casual expert
Ok,
now
that
you’ve
seen
5
examples
for
visualizaBons
of
listening
histories
that
approached
different
aspects
of
the
topic,
where
do
we
go
from
here?
111. RECOLLECTING
REMINISCING
RETRIEVING
REFLECTING
REMEMBERING Sellen, Whittaker: Beyond Total Capture: A
Constructive Critique of Lifelogging,
CACM, May 2010
VisualizaBon
is
nice
and
all,
but
there
is
more
that
we
can
do
with
these
histories.
It’s
nice
to
give
the
creators
of
these
histories
the
chance
to
recollect,
reminisce
and
so
on,
but
we
can
also
use
them
to
make
their
day-‐to-‐day
interacBon
with
music
easier
and
more
convenient.
112. In
these
last
few
minutes
of
my
talk
I
will
show
you
two
examples
of
how
to
use
this
data
in
other
areas.
113. One
problem
with
listening
to
music
is
that
there
a
mostly
only
two
ways
to
do
it:
You
either
manually
create
a
playlist
or
pick
an
album
or
have
it
done
fully
automaBcally.
The
former
makes
it
very
tedious
to
listen
to
music
(especially
on
the
go),
while
the
laXer
restricts
you
to
the
choice
of
the
machine
that
might
be
giving
you
the
same
songs
over
and
over
again
and
you
have
very
liXle
influence
on
that.
114. Rush
With
our
Rush-‐interacBon
technique
we
wanted
to
create
and
opBon
for
building
playlists
between
the
two
extremes
and
we
called
this
approach
‘repeated
recommendaBons’…
115. VIDEO
Rush
You
start
just
like
in
the
automaBc
case
with
a
hand-‐picked
seed
song
and
receive
a
set
of
five
recommendaBons
for
this
item.
Once
you
choose
once
of
these
items,
you
get
another
set
of
five
and
so
on
and
so
forth.
The
great
thing
about
this
approach
is
that
you
do
not
have
the
large
overhead
of
going
through
your
whole
collecBon
to
create
a
playlist,
but
sBll
have
much
more
freedom
than
in
the
purely
automaBc
case.
116. So
where
do
listening
histories
come
in
here?
First,
we
can
of
course
use
them
to
shape
the
recommended
items.
In
our
study
we
used
a
pre-‐defined
set
of
music
and
general
recommendaBons
from
last.fm
but
it
would
of
course
make
more
sense
to
adapt
the
recommendaBons
based
on
the
user’s
history….
117. …
second:
Five
items
is
not
a
lot,
so
it
is
difficult
to
choose
the
right
ones
in
order
not
to
frustrate
the
user.
Having
his
or
her
listening
history
available
means
that
we
can
automaBcally
remove
candidates
that
the
user
does
not
know
(and
would
not
be
very
helpful
in
this
scenario).
118. RECOLLECTING
Another
thing
that
you
can
do
when
working
with
listening
histories
is
use
them
for
rediscovery
of
music
that
you
forgot.
That
was
something
that
we
oIen
observed
when
people
used
one
of
the
visualizaBons
that
they
were
happy
to
find
some
song
or
arBst
that
they
had
forgoXen
about.
119. But
using
the
visualizaBons
is
an
explicit
acBvity
and
people
commonly
use
different
soIware
to
actually
listen
to
music.
So
in
this
last
project,
we
wanted
to
help
them
with
recollecBng
and
reminiscing
while
they
were
actually
listening
to
music.
120. So
we
decided
to
make
a
plugin
for
a
media
player.
Because
we
wanted
to
keep
it
useful
for
as
many
people
as
possible
we
chose
Songbird,
an
open
source
media
player
with
an
acBve
community,
that’s
available
for
Mac
and
Windows
instead
of
iTunes
or
the
Windows
Media
Player.
121. Our
idea
for
supporBng
rediscovery
was
based
on
the
idea
that
also
the
Tangle
visualizaBon
was
based
on:
Every
Bme
a
song
appears
in
a
listening
history
it
has
successors
and
predecessors.
And
this
order
of
songs
is
probably
important
for
the
listener,
not
always,
of
course,
but
at
least
someBmes.
So
the
idea
was
to
show
for
the
currently
playing
song
whatever
songs
appeared
before
and
aIer
it.
122. SongSlope
The
result
looks
like
this:
By
doing
what
they
would
have
done
anyway,
namely
listening
to
music,
users
automaBcally
receive
a
focused
glimpse
into
their
listening
past.
All
songs
before
and
aIer
are
displayed
and
they
can
switch
to
one
of
these
songs
simply
by
clicking
on
them.
123. SongSlope
…
and
users
can
also
switch
to
a
view
of
the
underlying
listening
sessions,
browse
through
them
or
listen
to
them
as
a
new
playlist.
124. Currently:
7,200 downloads
58 filled-out questionnaires
(40 partial)
We
had
a
lot
of
downloads
(as
I
said,
Songbird
has
a
very
acBve
community)
but
not
as
many
answers
to
the
quesBonnaire,
probably
because
we
had
no
pop-‐up
or
email
reminder
to
fill
it
out.
We
also
logged
the
relevant
aspects
of
the
user’s
interacBon
with
the
plug-‐in
(of
course,
only
aIer
they
agreed
to
that).
125. Use cases:
44.8% Re-discovering music
31.0% Generating playlists
We
were
especially
interested
in
what
people
used
it
for
and
found
that
almost
half
of
them
were
able
to
rediscover
music
with
it,
but
also
almost
a
third
used
it
for
creaBng
playlists
(or
relistening
to
old
playlists).
So
even
though
only
a
couple
of
people
answered
the
quesBonnaire
we
got
very
posiBve
feedback
from
them.
126. Ok,
so
where
does
that
leave
us
and
what
can
you
take
away
from
this
talk:
127. Listening
histories
are
today
mostly
used
for
recommendaBon.
But
as
they
are
a
type
of
personal
data
that
can
be
easily
collected
and
sBll
can
have
a
powerful
impact
into
people’s
lives
using
them
for
recommending
music
only
is
–
I
think
–
somewhat
of
a
waste.
We
can
do
much
more
with
them.
128. Screenshot
Of
LastLoop
playful casual expert
…
as
you’ve
seen:
We
can
visualize
this
informaBon
to
allow
people
to
reminisce
about
their
past
and
recollect
their
memories,
in
varying
degrees
of
complexity
and
for
various
approaches
to
the
topic…
129. And
beyond
navel-‐gazing
we
can
also
use
this
data
for
helping
people
with
listening
to
music:
We
can
use
listening
histories
to
improve
the
usability
on
mobile
devcies
for
quickly
and
conveniently
creaBng
personalized
playlists
on
the
go
or
to
add
value
by
lehng
people
painlessly
rediscover
music
while
listening
to
it
anyway.
130. Genre
……
Sub-Genre
Artists
Albums
Songs
Tags
So,
for
three
more
concrete
results
that
I
learned
while
working
this
topic:
It’s
probably
a
good
idea
to
use
a
Bmeline
as
the
central
metaphor
for
represenBng
personal
histories,
as
the
temporal
aspect
is
very
important
for
filing
this
data
into
one’s
personal
life
story.
Also,
abstracBons
such
as
genre
hierarchies
are
great
for
reducing
the
complexity
of
the
data
while
preserving
the
access
to
single
items.
131. 131
Second,
for
collecBng
results
from
casual
users
several
approaches
can
be
helpful:
We
had
quesBonnaires
that
popped
up
aIer
a
while
in
LastHistory,
we
tracked
relevant
interacBon
with
the
user’s
consent
to
learn
about
how
an
applicaBon
is
used
and
where
it
fails
(in
SongSlope)
and
finally,
the
feedback-‐buXon
that
we
had
in
LastLoop
allowed
for
impromptu
feedback
with
minimal
overhead.
132. Finally,
one
very
interesBng
data
source
that
we
tapped
when
creaBng
LastHistory
were
the
user’s
memories.
These
memories
can
give
context
and
meaning
to
plain
lists
of
songs
and
by
using
suitable
memory
triggers
it’s
possible
to
unearth
great
stories
and
understand
these
histories.
Depending
on
the
use
case,
visualizaBon
shouldn’t
underesBmate
the
value
of
having
a
real
person
sihng
in
front
of
the
machine.
133. I
think
the
central
part
is
that
these
histories
are
reflecBons
of
their
creators’
lives:
Music
accompanies
them
during
their
good
and
their
bad
Bmes,
their
triumphs
and
their
tragedies
and
forms
an
inseparable
bond
with
these
events.
But
what
they
are
lacking
are
the
tools
to
use
them
in
the
same
way
that
they
use
photos
for
reflecBng
about
their
past
and
making
sense
of
their
lives.
So
I
hope
my
work
is
a
first
step
towards
giving
this
data
back
to
the
people
who
created
it.
134. DOMINIKUS BAUR
UNIVERSITY OF dominikus.baur@ifi.lmu.de
MUNICH (LMU), twitter: @dominikus
GERMANY
Thank
you!