The Echo Nest at Music and Bits, October 21 2009

I am losing my voice.
I am sorry.
I am normally louder than this.
I also added text to the pictures.

Thursday, October 22, 2009

A Short (Personal) History of
Computers Listening to Music

1999-2009


I was a musician for a while.
Electronic music.
“Intelligent dance music”
(worst genre name ever)


“Fish / Cut bait”

Handheld-music (1998-2001)
I made my own software to make music


“Fish / Cut bait”

Handheld-music (1998-2001)
Did it make me a better musician? Deﬁnitely not.


It was 1999. Lots of stuff was happening.


I learned about music from reading web sites.
Forums, mailing lists.


You could now download a song faster than real time.
I ﬁgured things would change quick.


So I went to grad school.
I studied information retrieval, language processing.


Columbia University, NYC

MIT Media Lab
ﬁnishing my dissertation


People were starting to apply IR techniques to music.
Audio ﬁles are treated like text.
FFT frames became words
Songs became “documents”


There’s a problem with that.
Just because you can convert an mp3 to #s
doesn’t mean you understand it.


“Music IR” was born.
The applications are varied, but most
have nothing to do with music.


Retrieving Music by Rhythmic Similarity

1.5

110 bpm

112 bpm
1
squared Euclidean distance

114 bpm
120 bpm
122 bpm 116 bpm

124 bpm

0.5 126 bpm

128 bpm

130 bpm
0
130 128 126 124 122 120 118 116 114 112 110
Tempo (bpm)

Figure 5. Euclidean Distance vs. Tempo

The worst offender: “Genre Identiﬁcation”
Countless PhDs on this useless task.
Trying to teach a computer a marketing construct.


Show of hands:

Is Bjork “electronic, pop, jazz”?


At MIT I convinced someone to buy lots of computers


And tried to ﬁgure out how to get music
into music analysis


Simple things like detecting holiday music
is very hard.


I decided if I could get a computer to make
holiday music,

We could claim we understand it.


This is automatically generated holiday music
Music Acquisition (2001-)
based on listening to 1,000 Christmas songs


It should be a funny joke that you can
run statistics of millions of things
and “understand it.”


I built Eigenradio in 2003 to show people
What computers hear when they hear music


There’s obviously so much more to music
than the audio signal
and that other stuff
is probably more important


My brother makes music with sine waves
and nothing else
and gets a 9.7 on Pitchfork.
This is fascinating!


My brother makes music with sine waves
and nothing else
and gets a 9.7 on Pitchfork.
This is fascinating!
Were the sine waves that good?


Review Regression

It turns out if you understand language
and audio
at the same time
you start learning a lot more.


Here we predict ratings on All Music Guide
and Pitchfork
By listening to the audio and reading about the artist.


Audio alone was terrible
Text alone was better than audio
Both together were the best.


100 100
12 10
80 80
Pitchfork Ratings

Pitchfork Ratings
10
8
60 8 60
6
40 6 40
4
4
20 20 2
2

2 4 6 8 2 4 6 8
AMG Ratings Randomly selected AMG Ratings

100
.147 120 .127 6
Audio−derived Ratings

Audio−derived Ratings
8
80

6
[.080] 100

80 60
[.082] 5
4
60 3
4 40
40 2
2 20
20 1

2 4 6 8 20 40 60 80 100
AMG Ratings Pitchfork Ratings


I became interested in more ridiculous questions:
“Can we ﬁnd the saddest song in the world?”


So I started a company in 2005
with my co-founder Tristan, also at the Lab.


Tristan is a DSP “machine listening” expert
and I handled the text side


MAGIC


Why does the Echo Nest exist?


The best music experience is still very manual.
I am still reading about music, not using a recommender.


& the act of listening to music is easier than ever


But data is hard.
Most designers make very bad decisions
because their tools are inefﬁcient.


Collaborative ﬁltering (X who did Y also did Z)
is so easy to make; but it’s also so terrible.


Collaborative ﬁltering (X who did Y also did Z)
is so easy to make; but it’s also so terrible.
The SQL join is destroying music.


In 2005 we modeled the worst case scenario:

In which collaborative ﬁltering was the only way
for an artist to get noticed.

The popular ones would eat the unknown ones alive.

3 sets of 3 artists each remained.


Set A Set B Set C
Britney Spears Alice in Chains Chris Isaak
Backstreet Boys Korn Bob Dylan
Cristina Aguilera Faith no More Crowded House


So the Echo Nest gives everyone great data.
They can decide on their own how to show it.


The Echo Nest 2005

Somerville, MA USA
2 people
2 computers
Lots of ideas
1m documents
10,000 artists
100,000 songs
0 public facing sites


The Echo Nest 2009

Somerville, MA USA
20 people
200 computers
Lots of products
5bn documents
1,000,000 artists
many millions of songs
0 public facing sites


What We Do


“Know everything about music and listeners.”


“Give (and sell) great data to everyone.”


“Give (and sell) great data to everyone.”
“Do it automatically with no bias, on everything.”


Customers Crawling Code

NLP DSP Machine Learning


Artist Data Song Data Listener Data
• Tag Clouds • Similar Songs • demographics
• Similar Artists • Tempo - age, gender, location
• Analytics • Key • psychographics
Familiarity Mode - preferences, lifestyle
• •
• music preference
• Hotttnesss • Time Signature
Blogs Beats • listening patterns
• •
News Downbeats • tastemaker proﬁling
• •
- writers, bloggers
• Reviews • Segments
• Audio • Timbre
• Video • Pitch
• Proﬁle Sites • Loudness
• Misspellings • Sections
• Aliases


We have a lot of data and
we have a lot of products.
We sell mostly to social networks, labels;
video games; PR ﬁrms; musicians


Artist metrics

Acoustic
Similarity analysis

Feeds
Remix

Metadata
Search / Tags

Predictive Recommendation
analytics


The reason we are special is 2 things:

Scale and Platform


Our scale is limitless.
We have hundreds of computers
We always do our computation on everything.
We can learn about new music very quickly.


Scale

All Music Guide Pandora The Echo Nest

known artists 280,000 80,000 1,000,000

years to get there 18 8 1

time to understand
1 week 1 day <1 minute
one album

cost to understand
$400 $40 $0.001
one album


Our platform is huge.
We have thousands of “free” developers using our API
Our customers use the same platform
So do we.


Platform


We sell two main products:

Fanalytics is a predictive analytics toolset for artists

The Knowledge is a dynamic metadata service
(recommendation, feeds, data)
for web sites


Fanalytics lets artists and labels get a view
into the world of online music
We recommend blogs for artists
We show predicted analytics on activity


Predictive
analytics

Artist metrics


We also maintain a popular open source
remixing community and code base
so people can make awesome free
mashups, remixes, web sites using our tech

Not much of a business but we love it.


Remix


“DonkDJ.com” was made using Remix
It automatically “donks” (ask someone what this means)
any song you upload


Morecowbell.dj adds cowbell to any song

This Is My Jam was a pre-Muxtape (by one day)
mixtape sharing site that only let you use 30s samples
and made a total mess of the output.

Like I said, not much of a business.


We also have artists using Remix
-- our data is now powering some next generation
electronic music


I’ve always wanted to hear Michael
Jackson trying to sing Amerie’s “One
Thing” automatically by comparing
timbre, pitch and loudness distances.

-B.L.


James Brown... FOREVER.


Remix also works on video


Let’s hear Daft Punk’s “Revolution 909”
played by a ﬁght scene from Undefeatable!
-Y.A.


Our analysis data powers a lot of visualizers and
video games (rhythm games on your own MP3s)


Acoustic
analysis


The Knowledge is a much better music data service
Customers can subscribe to constantly-updated
similarity, metadata, feeds, recommendations, etc


Our similarity and recommendation data
is some of the best, because we use so many sources
and we know about all artists even if they are tiny


Similarity

Feeds


Since our similarity is based on so many features:
popularity, audio analysis, text analysis,
structured metadata, inﬂuences, ...


We provide our customers with the knobs
and let them decide what is important for the task.


We provide our customers with the knobs
and let them decide what is important for the task.
We do not give a “single answer.”
There is no single answer.


Similarity


We can build paths between artists on any vector


Acoustic
Similarity analysis

Search / Tags


Our future:


1. Listener analytics


We’ve been running large scale data mining
on millions of listeners to help with analytics,
for example a gender predictor based on your music taste


Here’s the basis vectors; strongest correlators of gender:


Male Female

Pet Shop Boys Eternal

Fort Minor Metro Station

Justice Gackt

Mike Oldﬁeld Paolo Nutini

U2 London after Midnight


2. More musicians to use our remix tools


(I’ve noticed the better you are with computers,
the worse your music is. This may just be me)


100%

75%
Music goodness

50%

25%

0%
nothing not much a little somewhat pretty good expert dork prime
Computers know-how


3. Search anything APIs


We will soon make all of our acoustic data
available for searching and browsing
(right now it has to be your content):

“Find me a drum hit in this collection
that sounds like the break in ‘Single Ladies’”


Combined with Remix this will allow anyone
to compose music that uses all music in the world


>> from echonest import search
>> segments = search.query(“voice”, soundsLike=”bjork”, pitch=”F#”)
>> len(segments)
65706
>> new_song = random.shuffle(segments).write(“bjork2009.mp3”)


To wrap up:

1. Don’t trust computers


To wrap up:

2. But trust us, really


To wrap up:

2. But trust us, really
3. Sorry I can’t speak very well


The Echo Nest at Music and Bits, October 21 2009

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Andere mochten auch

Andere mochten auch (17)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

The Echo Nest at Music and Bits, October 21 2009