Did you ever get that feeling when a random song pops into your brain and you can’t get rid of it? Well, that happened to me recently and I couldn’t even remember the title of the damn song! In this talk, I want to share with you the story of how I was able to recover the details of the song by navigating some music-related APIs using JavaScript, Node.js and the magic of async iterators!
11. Let me introduce myself first...
I'm Luciano ( 🍕🍝) 👋
Senior Architect @ fourTheorem (Dublin )
9
12. Let me introduce myself first...
I'm Luciano ( 🍕🍝) 👋
Senior Architect @ fourTheorem (Dublin )
nodejsdp.link
Co-Author of Node.js Design Patterns 👉
9
13. Let me introduce myself first...
I'm Luciano ( 🍕🍝) 👋
Senior Architect @ fourTheorem (Dublin )
nodejsdp.link
Co-Author of Node.js Design Patterns 👉
Connect with me:
(blog)
(twitter)
(twitch)
(github)
loige.co
@loige
loige
lmammino 9
14. We are business focused technologists that
deliver.
| |
Accelerated Serverless AI as a Service Platform Modernisation
We are hiring: do you want to ?
work with us
loige 10
26. loige
Let's give it a shot
curl "http://ws.audioscrobbler.com/2.0/?
method=user.getrecenttracks&user=loige&api_key
=${API_KEY}&format=json" | jq .
19
32. loige
We are getting a "paginated" response
with 50 tracks per page
but there are 51 here! 🤔
23
33. loige
We are getting a "paginated" response
with 50 tracks per page
but there are 51 here! 🤔
(let's ignore this for now...)
23
34. loige
We are getting a "paginated" response
with 50 tracks per page
but there are 51 here! 🤔
How do we fetch the next pages?
(let's ignore this for now...)
23
48. loige
* Note that page size
here is 10 tracks per
page
Every page has a song with undefined time...
This is the song I am currently listening to!
It appears at the top of every page.
29
49. loige
* Note that page size
here is 10 tracks per
page
Sometimes there are duplicated tracks
between pages... 😨
29
62. loige
...*
tracks (newest to oldest)
34
Page1 before t1
(page 1 "to" t1)
t1
* we are done when we get an empty page (or num pages is 1)
63. loige
...*
tracks (newest to oldest)
34
Page1 before t1
(page 1 "to" t1)
t1 t2
* we are done when we get an empty page (or num pages is 1)
64. loige
...*
tracks (newest to oldest)
34
Page1 before t1
(page 1 "to" t1)
t1 t2
before t2
(page 1 "to" t2)
* we are done when we get an empty page (or num pages is 1)
65. let to
while (true) {
const query = querystring.stringify({
method: 'user.getrecenttracks',
user: 'loige',
api_key: process.env.API_KEY,
format: 'json',
limit: '10',
to
})
const url = `https://ws.audioscrobbler.com/2.0/?${query}`
const response = await axios.get(url)
const tracks = response.data.recenttracks.track
console.log(
`--- ↓ page to ${to}`,
`remaining pages: ${response.data.recenttracks['@attr'].totalPages} ---`
)
for (const track of tracks) {
console.log(track.date?.uts, `${track.artist['#text']} - ${track.name}`)
}
if (response.data.recenttracks['@attr'].totalPages <= 1) {
break // it's the last page!
}
const lastTrackInPage = tracks[tracks.length - 1]
to = lastTrackInPage.date.uts
} loige 35
66. let to
while (true) {
const query = querystring.stringify({
method: 'user.getrecenttracks',
user: 'loige',
api_key: process.env.API_KEY,
format: 'json',
limit: '10',
to
})
const url = `https://ws.audioscrobbler.com/2.0/?${query}`
const response = await axios.get(url)
const tracks = response.data.recenttracks.track
console.log(
`--- ↓ page to ${to}`,
`remaining pages: ${response.data.recenttracks['@attr'].totalPages} ---`
)
for (const track of tracks) {
console.log(track.date?.uts, `${track.artist['#text']} - ${track.name}`)
}
if (response.data.recenttracks['@attr'].totalPages <= 1) {
break // it's the last page!
}
const lastTrackInPage = tracks[tracks.length - 1]
to = lastTrackInPage.date.uts
} loige 35
67. let to
while (true) {
const query = querystring.stringify({
method: 'user.getrecenttracks',
user: 'loige',
api_key: process.env.API_KEY,
format: 'json',
limit: '10',
to
})
const url = `https://ws.audioscrobbler.com/2.0/?${query}`
const response = await axios.get(url)
const tracks = response.data.recenttracks.track
console.log(
`--- ↓ page to ${to}`,
`remaining pages: ${response.data.recenttracks['@attr'].totalPages} ---`
)
for (const track of tracks) {
console.log(track.date?.uts, `${track.artist['#text']} - ${track.name}`)
}
if (response.data.recenttracks['@attr'].totalPages <= 1) {
break // it's the last page!
}
const lastTrackInPage = tracks[tracks.length - 1]
to = lastTrackInPage.date.uts
} loige 35
68. let to
while (true) {
const query = querystring.stringify({
method: 'user.getrecenttracks',
user: 'loige',
api_key: process.env.API_KEY,
format: 'json',
limit: '10',
to
})
const url = `https://ws.audioscrobbler.com/2.0/?${query}`
const response = await axios.get(url)
const tracks = response.data.recenttracks.track
console.log(
`--- ↓ page to ${to}`,
`remaining pages: ${response.data.recenttracks['@attr'].totalPages} ---`
)
for (const track of tracks) {
console.log(track.date?.uts, `${track.artist['#text']} - ${track.name}`)
}
if (response.data.recenttracks['@attr'].totalPages <= 1) {
break // it's the last page!
}
const lastTrackInPage = tracks[tracks.length - 1]
to = lastTrackInPage.date.uts
} loige 35
69. let to
while (true) {
const query = querystring.stringify({
method: 'user.getrecenttracks',
user: 'loige',
api_key: process.env.API_KEY,
format: 'json',
limit: '10',
to
})
const url = `https://ws.audioscrobbler.com/2.0/?${query}`
const response = await axios.get(url)
const tracks = response.data.recenttracks.track
console.log(
`--- ↓ page to ${to}`,
`remaining pages: ${response.data.recenttracks['@attr'].totalPages} ---`
)
for (const track of tracks) {
console.log(track.date?.uts, `${track.artist['#text']} - ${track.name}`)
}
if (response.data.recenttracks['@attr'].totalPages <= 1) {
break // it's the last page!
}
const lastTrackInPage = tracks[tracks.length - 1]
to = lastTrackInPage.date.uts
} loige 35
78. const reader = LastFmRecentTracks({
apikey: process.env.API_KEY,
user: 'loige'
})
// ASYNC ITERATORS!
for await (const page of reader) {
/* ... */
}
// ... do more stuff when all the data is consumed
loige 43
79. const reader = LastFmRecentTracks({
apikey: process.env.API_KEY,
user: 'loige'
})
// ASYNC ITERATORS WITH ERROR HANDLING!
try {
for await (const page of reader) {
/* ... */
}
} catch (err) {
// handle errors
}
// ... do more stuff when all the data is consumed loige 44
80. How can we build an async iterator?
🧐
loige 45
81. Meet the iteration protocols!
loige
loige.co/javascript-iterator-patterns
46
82. The iterator protocol
An object is an iterator if it has a next() method.
Every time you call it, it returns an object with
the keys done (boolean) and value.
loige 47
83. function createCountdown (from) {
let nextVal = from
return {
next () {
if (nextVal < 0) {
return { done: true }
}
return {
done: false,
value: nextVal--
}
}
}
} loige 48
84. function createCountdown (from) {
let nextVal = from
return {
next () {
if (nextVal < 0) {
return { done: true }
}
return {
done: false,
value: nextVal--
}
}
}
} loige 48
85. function createCountdown (from) {
let nextVal = from
return {
next () {
if (nextVal < 0) {
return { done: true }
}
return {
done: false,
value: nextVal--
}
}
}
} loige 48
86. function createCountdown (from) {
let nextVal = from
return {
next () {
if (nextVal < 0) {
return { done: true }
}
return {
done: false,
value: nextVal--
}
}
}
} loige 48
87. function createCountdown (from) {
let nextVal = from
return {
next () {
if (nextVal < 0) {
return { done: true }
}
return {
done: false,
value: nextVal--
}
}
}
} loige 48
94. The iterable protocol
An object is iterable if it implements the
@@iterator* method, a zero-argument function
that returns an iterator.
loige
*Symbol.iterator
53
95. function createCountdown (from) {
let nextVal = from
return {
[Symbol.iterator]: () => ({
next () {
if (nextVal < 0) {
return { done: true }
}
return { done: false, value: nextVal-- }
}
})
}
}
loige 54
96. function createCountdown (from) {
let nextVal = from
return {
[Symbol.iterator]: () => ({
next () {
if (nextVal < 0) {
return { done: true }
}
return { done: false, value: nextVal-- }
}
})
}
}
loige 54
97. function createCountdown (from) {
let nextVal = from
return {
[Symbol.iterator]: () => ({
next () {
if (nextVal < 0) {
return { done: true }
}
return { done: false, value: nextVal-- }
}
})
}
}
loige 54
98. function createCountdown (from) {
return {
[Symbol.iterator]: function * () {
for (let i = from; i >= 0; i--) {
yield i
}
}
}
}
loige 55
99. const countdown = createCountdown(3)
for (const value of countdown) {
console.log(value)
}
// 3
// 2
// 1
// 0
loige 56
100. OK. So far this is all synchronous iteration.
What about async? 🙄
loige 57
101. The async iterator protocol
An object is an async iterator if it has a next()
method. Every time you call it, it returns a
promise that resolves to an object with the keys
done (boolean) and value.
loige 58
102. import { setTimeout } from 'timers/promises'
function createAsyncCountdown (from, delay = 1000) {
let nextVal = from
return {
async next () {
await setTimeout(delay)
if (nextVal < 0) {
return { done: true }
}
return { done: false, value: nextVal-- }
}
}
} loige 59
103. import { setTimeout } from 'timers/promises'
function createAsyncCountdown (from, delay = 1000) {
let nextVal = from
return {
async next () {
await setTimeout(delay)
if (nextVal < 0) {
return { done: true }
}
return { done: false, value: nextVal-- }
}
}
} loige 59
104. import { setTimeout } from 'timers/promises'
function createAsyncCountdown (from, delay = 1000) {
let nextVal = from
return {
async next () {
await setTimeout(delay)
if (nextVal < 0) {
return { done: true }
}
return { done: false, value: nextVal-- }
}
}
} loige 59
108. import { setTimeout } from 'timers/promises'
// async generators "produce" async iterators!
async function * createAsyncCountdown (from, delay = 1000) {
for (let i = from; i >= 0; i--) {
await setTimeout(delay)
yield i
}
}
loige 62
109. import { setTimeout } from 'timers/promises'
// async generators "produce" async iterators!
async function * createAsyncCountdown (from, delay = 1000) {
for (let i = from; i >= 0; i--) {
await setTimeout(delay)
yield i
}
}
loige 62
110. import { setTimeout } from 'timers/promises'
// async generators "produce" async iterators!
async function * createAsyncCountdown (from, delay = 1000) {
for (let i = from; i >= 0; i--) {
await setTimeout(delay)
yield i
}
}
loige 62
111. The async iterable protocol
An object is an async iterable if it implements
the @@asyncIterator* method, a zero-argument
function that returns an async iterator.
loige
*Symbol.asyncIterator
63
112. import { setTimeout } from 'timers/promises'
function createAsyncCountdown (from, delay = 1000) {
return {
[Symbol.asyncIterator]: async function * () {
for (let i = from; i >= 0; i--) {
await setTimeout(delay)
yield i
}
}
}
}
loige 64
113. import { setTimeout } from 'timers/promises'
function createAsyncCountdown (from, delay = 1000) {
return {
[Symbol.asyncIterator]: async function * () {
for (let i = from; i >= 0; i--) {
await setTimeout(delay)
yield i
}
}
}
}
loige 64
114. const countdown = createAsyncCountdown(3)
for await (const value of countdown) {
console.log(value)
}
loige 65
115. const countdown = createAsyncCountdown(3)
for await (const value of countdown) {
console.log(value)
}
loige 65
116. Now we know how to make our
LastFmRecentTracks an Async Iterable 🤩
loige 66
117. function createLastFmRecentTracks (apiKey, user) {
return {
[Symbol.asyncIterator]: async function * () {
let to
while (true) {
const query = querystring.stringify({
method: 'user.getrecenttracks',
user,
api_key: apiKey,
format: 'json',
to
})
const url = `https://ws.audioscrobbler.com/2.0/?${query}`
const response = await axios.get(url)
const tracks = response.data.recenttracks.track
yield tracks
if (response.data.recenttracks['@attr'].totalPages <= 1) {
break // it's the last page!
}
const lastTrackInPage = tracks[tracks.length - 1]
to = lastTrackInPage.date.uts
}
}
}
}
loige 67
118. function createLastFmRecentTracks (apiKey, user) {
return {
[Symbol.asyncIterator]: async function * () {
let to
while (true) {
const query = querystring.stringify({
method: 'user.getrecenttracks',
user,
api_key: apiKey,
format: 'json',
to
})
const url = `https://ws.audioscrobbler.com/2.0/?${query}`
const response = await axios.get(url)
const tracks = response.data.recenttracks.track
yield tracks
if (response.data.recenttracks['@attr'].totalPages <= 1) {
break // it's the last page!
}
const lastTrackInPage = tracks[tracks.length - 1]
to = lastTrackInPage.date.uts
}
}
}
}
loige 67
119. function createLastFmRecentTracks (apiKey, user) {
return {
[Symbol.asyncIterator]: async function * () {
let to
while (true) {
const query = querystring.stringify({
method: 'user.getrecenttracks',
user,
api_key: apiKey,
format: 'json',
to
})
const url = `https://ws.audioscrobbler.com/2.0/?${query}`
const response = await axios.get(url)
const tracks = response.data.recenttracks.track
yield tracks
if (response.data.recenttracks['@attr'].totalPages <= 1) {
break // it's the last page!
}
const lastTrackInPage = tracks[tracks.length - 1]
to = lastTrackInPage.date.uts
}
}
}
}
loige 67
120. const recentTracks = createLastFmRecentTracks(
process.env.API_KEY,
'loige'
)
for await (const page of recentTracks) {
console.log(page)
}
loige 68
121. Let's search for all the songs that contain the
word "dark" in their title! 🧐
loige 69
122. async function main () {
const recentTracks = createLastFmRecentTracks(
process.env.API_KEY,
'loige'
)
for await (const page of recentTracks) {
for (const track of page) {
if (track.name.toLowerCase().includes('dark')) {
console.log(`${track.artist['#text']} - ${track.name}`)
}
}
}
}
loige 70
125. For a more serious package that allows you to
fetch data from Last.fm:
loige
npm install scrobbles
72
126. Cover picture by on
Thanks to Jacek Spera, , , ,
for reviews and suggestions.
Eric Nopanen Unsplash
@eoins @pelger @gbinside
@ManuEomm
-
loige.link/iter-sails loige.link/async-it-code
for await (const _ of createAsyncCountdown(1_000_000)) {
console.log("THANK YOU! 😍")
}
loige
nodejsdp.link
73