1. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
MUSICDISCOBERRY
the bittersweet high-hanging fruit
Òscar Celma

2. “The Paradox of Choice: Why More Is Less” (Barry Schwartz, 2004)

3. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Help! I’m stuck in the head
The limited reach of music recommendation
Popularity
48% of recommendations
0% of
recommendations
52% of recommendations
83 Artists 6,659 Artists 239,798 Artists
Sales Rank
Last.fm similar artists

4. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
WHAT WHO WHY WHEN WHERE

5. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
WHAT WHO WHY WHEN WHERE

6. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Music Similarity
● Content-based
● Usage-based
● Social-based
● Hybrid (combination)

7. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Music Similarity
● Content-based
● Usage-based
● Social-based
● Hybrid (combination)

8. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Audio Content-Based Analysis
Song: “Call Me Maybe” Carly Rae Jepsen

9. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Audio Content-Based Analysis
Song: “Call Me Maybe” Carly Rae Jepsen
Rhythm: beat detection, BPMs
Timbre (MFCC)
Harmony (Chromagram)

10. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Audio Content-Based Analysis
Song: “Call Me Maybe” Carly Rae Jepsen
Rhythm: beat detection, BPMs
Timbre (MFCC)
Harmony (Chromagram)

11. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Audio Content-Based Analysis
Song: “Call Me Maybe” Carly Rae Jepsen
Rhythm: beat detection, BPMs
Timbre (MFCC)
Harmony (Chromagram)

12. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Audio Content-Based Analysis
Song: “Call Me Maybe” Carly Rae Jepsen
...as well as
Rhythm: beat detection, BPMs
loudness
danceability
studio vs. live
Timbre (MFCC) structure (ABABCA)
genre (pop)
mood (euphoric, energetic)
Harmony (Chromagram)

13. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Audio Content-Based Similarity
Song 1: “Call Me Maybe” Carly Rae Jepsen Song 2
Rhythm (beat detection, BPMs) Rhythm (beat detection, BPMs)
Timbre (MFCC) Timbre (MFCC)
Harmony (Chromagram) Harmony (Chromagram)

14. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Audio Content-Based Similarity
Song 1: “Call Me Maybe” Carly Rae Jepsen Song 2
Rhythm (beat detection, BPMs)
sim(Song1, Song2) = ???
Rhythm (beat detection, BPMs)
EMD & k-means
KL & GMM
Timbre (MFCC) L2 & PCA (MFCC)
Timbre
Adhoc
...
Harmony (Chromagram) Harmony (Chromagram)

15. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Audio Content-Based Similarity

16. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Music Similarity
● Content-based
● Usage-based
● Social-based
● Hybrid (combination)

17. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Usage-Based
● Collaborative filtering
● “people who listen to X also listen to Y”
● Matrix Factorization
● SVD, NMF, ...

18. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Usage-Based The Beatles Example

19. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Usage-Based The Beatles Example

20. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Long Tail in Music
● From Hits to Niches
how many clicks?

21. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Long Tail in Music
● From Hits to Niches
● Audio Content-Based similarity
– 2 clicks
● Usage-Based (last.fm)
– 5 clicks (!)
(yet, perceived quality is higher for usage-based)

22. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Music Similarity
● Content-based
● Usage-based
● Social-based
● Hybrid (combination)

23. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Social-Based
Content Reviews Blogs Social Tags Bios Playlists
thrash
heavy metal
Weird Edgy
concert
gothic 90s
rock Loud

24. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Social-Based The Beatles Example

25. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Social-Based The Beatles Example

26. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Social-Based Black Metal Tag

27. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Social-Based Black Metal Tag

28. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Social-Based Black Metal Tag
???

29. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Social-Based Black Metal Tag
http://www.last.fm/bestof/10years/artists/tag/black+metal

30. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Music Similarity
● Content-based
● Usage-based
● Social-based
● Hybrid (combination)

31. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Hybrid
● Weighted (linear combination)
● E.g Usage * 0.4 + Social * 0.4 + Content * 0.2
● Cascade
●
E.g 1st apply Social, then reorder by Content-Based
● (...)

32. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Hybrid Music Similarity

33. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
WHAT WHO WHY WHEN WHERE
● Consider all music including new, and not-so popular

34. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
WHAT WHO WHY WHEN WHERE
● Consider all music including new, and not-so popular

35. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
WHAT WHO WHY WHEN WHERE
● Consider all music including new, and not-so popular

36. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
WHAT WHO WHY WHEN WHERE

37. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
WHAT WHO WHY WHEN WHERE

38. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Radiohead similar artists
“If you like Radiohead you might like this article” (Celma & Lamere). AI Magazine, 2011

39. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Radiohead similar artists
? ? ?
Social-Based: Using Artist Tags
Content-Based: Audio Similarity (artist top tracks)
Usage-Based: CF
Using Last.fm-360K dataset (google: oscar celma datasets)

40. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Radiohead similar artists

41. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Recommendation
“People who like Radiohead also like X”
vs.
“Paul likes Radiohead, so he might also like Y”

42. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Recommendation
● To whom are we recommending? Phoenix-2. UK (2006)

43. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
lamere @ last.fm

44. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
lamere @ last.fm
● Clustering lamere top-20 artists (using tags)

45. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
lamere @ last.fm
● Clustering lamere top-20 artists (using tags)

46. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
@lamere - Radiohead
9

47. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
@lamere - Radiohead
9
vs. Radiohead similar artists...

48. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
@lamere - Radiohead
9
vs. Radiohead similar artists...

49. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
@lamere - Radiohead
9
vs. Radiohead similar artists...

50. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
@lamere - Radiohead
9
vs. Radiohead similar artists...

51. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
@lamere - Radiohead
9
vs. Radiohead similar artists...
Novel & Relevant music for Paul

52. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
WHAT WHO WHY WHEN WHERE

53. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Novelty & Relevance

54. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Novelty & Relevance

55. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Novelty & Relevance

56. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Novelty & Relevance

57. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Novelty & Relevance

58. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Novelty & Relevance

59. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Novelty & Relevance
Emitt Rhodes

60. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Novelty & Relevance
WHY as important as WHAT

61. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Novelty & Relevance
WHY as important as WHAT

62. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Novelty & Relevance
WHY as important as WHAT

63. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Novelty & Relevance
WHY as important as WHAT

64. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
Novelty & Relevance
SIMILAR ARTISTS: SIMILAR ARTISTS: SIMILAR ARTISTS:
The Roulettes The Idle Race Emitt Rhodes
The Fourmost The Flower Pot Men Hotlegs

65. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
WHAT WHO WHY WHEN WHERE

66. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
WHAT WHO WHY WHEN WHERE
?
Emitt Rhodes

67. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
WHAT WHO WHY WHEN WHERE

68. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
WHAT WHO WHY WHEN WHERE

69. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
WHAT WHO WHY WHEN WHERE

70. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
WHAT WHO WHY WHEN WHERE

71. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
WHAT WHO WHY WHEN WHERE
● Listener’s context

72. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
WHAT WHO WHY WHEN WHERE
● Listener’s context

73. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
WHAT WHO WHY WHEN WHERE
● Listener’s context

74. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
WHAT WHO WHY WHEN WHERE
● Listener’s context
● Working
● Driving
● Workout
● Studying
● Relaxing
● How to get user's attention?

75. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
WHAT WHO WHY WHEN WHERE

76. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
The Whole Picture

77. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
The Whole Picture

78. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma

79. recsys meetup 2012 @ berlin | musicdiscoberry | @ocelma
@ocelma
http://ocelma.net
MUSICDISCOBERRY
the bittersweet high-hanging fruit