Hey ,
I'm thrilled to help you learn JavaScript. Unfortunately, you've landed on a page where you cannot access with your current purchase.
Please upgrade (use this link) access this content.
I'm super eager to help you learn more!
Let’s say you want to fetch all items, but the API doesn’t tell you what the final page is. In this case, you’ll need to fetch the items page by page, until you get the final page.
The first thing you’ll do is to send your first request.
We’re going to fetch Sindre Sordus’s repositories for this lesson. Here’s how you can fetch the first 100 repositories.
fetch('https://api.github.com/users/sindresorhus/repos?per_page=100')
.then(r => r.json())
.then(repos => {
console.log(repos)
})
If the server gives us less than 100 repositories, we know we don’t need to make another request. We know we have all the repositories.
If we have all the repos, we’re done. We can return repos to use it in another then call.
fetch('https://api.github.com/users/sindresorhus/repos?per_page=100')
.then(r => r.json())
.then(repos => {
if (repos.length < 100) {
// We got all repos. No need to fetch any more
return repos
} else {
// Need to fetch more repos
}
})
.then(repos => { /* Do something with repos! */ })
We need to make a second request if the server responds with 100 repositories. This is because we don’t know if there’s anything else. It’s likely there are more repositories to fetch.
We can fetch the second page by setting the page parameter to 2. This parameter might be different from API to API. Make sure you check the documentation for the API you’re working with.
fetch('https://api.github.com/users/sindresorhus/repos?per_page=100')
.then(r => r.json())
.then(repos => {
if (repos.length < 100) {
return repos
} else {
return fetch('https://api.github.com/users/sindresorhus/repos?per_page=100&page=2')
.then(r => r.json())
.then(repos2 => {
// Do something with second page of repos
})
}
})
We want the full list of repositories in the third then call. This means we must add the second list of repositories to the first list.
fetch('https://api.github.com/users/sindresorhus/repos?per_page=100')
.then(r => r.json())
.then(repos => {
if (repos.length < 100) {
return repos
} else {
return fetch('https://api.github.com/users/sindresorhus/repos?per_page=100&page=2')
.then(r => r.json())
.then(repos2 => {
repos = repos.concat(repos2)
})
}
})
.then(repos => { /* Do something with repos! */})
We know we’re done if the server gives us less than 100 repositories. Here, we can return the concatenated repos.
We also know we’re not done if the server gives us 100 repositories. We need to send another request. This process goes on and on until the server sends back less than 100 repositories.
fetch('https://api.github.com/users/sindresorhus/repos?per_page=100')
.then(r => r.json())
.then(repos => {
if (repos.length < 100) {
return repos
} else {
return fetch('https://api.github.com/users/sindresorhus/repos?per_page=100&page=2')
.then(r => r.json())
.then(repos2 => {
repos = repos.concat(repos2)
if (repos2.length < 100) {
return repos
} else {
return fetch('https://api.github.com/users/sindresorhus/repos?per_page=100&page=3')
.then(r => r.json())
.then(repos3 => { /* ... */})
}
})
}
})
A recursive function is a function that calls itself over and over, until it doesn’t need to call itself. This means recursive functions usually have an if/else statement built into them.
const recursiveFunction = _ => {
if (someCondition) {
return someValue
} else {
return recursiveFunction()
}
}
Let’s build a recursive function together.
First, we know that we want to create a recursive function. Let’s call it recursiveFetch. This function should return a fetch promise.
const recursiveFetch = _ => {
return fetch('some-link')
.then(r => r.json())
}
We need to pass a link into the recursiveFetch function. This link should be the endpoint to Sindre’s repositories.
const recursiveFetch = link => {
return fetch(link)
.then(r => r.json())
}
// Usage
recursiveFetch('https://api.github.com/users/sindresorhus/repos?per_page=100')
.then(repos => console.log(repos))
If the server returns less than 100 repositories, we know we’re done. We’ll return repos.
const recursiveFetch = link => {
return fetch(link)
.then(r => r.json())
.then(repos => {
if (repos.length < 100) {
return repos
} else {
// Fetch second page
}
})
}
// Usage
recursiveFetch('https://api.github.com/users/sindresorhus/repos?per_page=100')
.then(repos => console.log(repos))
Here, recursiveFetch needs to know we’re expecting 100 items. We can let it know by adding a perPage argument.
const recursiveFetch = (link, perPage) => {
return fetch(link)
.then(r => r.json())
.then(repos => {
if (repos.length < perPage) {
return repos
} else {
// Fetch second page
}
})
}
// Usage
recursiveFetch('https://api.github.com/users/sindresorhus/repos?per_page=100', 100)
.then(repos => console.log(repos))
recursiveFetch already knows we’re expecting 100 items. It’s kind of silly to pass the ?per_page=100 query string into the link.
We can tell recursiveFetch to use perPage to fetch the correct number of items per request.
const recursiveFetch = (link, perPage) => {
return fetch(`${link}?per_page=${perPage}`)
.then(r => r.json())
.then(repos => {
if (repos.length < perPage) {
return repos
} else {
// Fetch second page
}
})
}
// Usage
recursiveFetch('https://api.github.com/users/sindresorhus/repos', 100)
.then(repos => console.log(repos))
If the server gives us 100 repositories, we know we need to make another fetch request. We can use the same link and perPage values in this request, but we need to add a new page parameter.
const recursiveFetch = (link, perPage) => {
return fetch(`${link}?per_page=${perPage}`)
.then(r => r.json())
.then(repos => {
if (repos.length < perPage) {
return repos
} else {
fetch(`${link}?per_page=${perPage}&page=2`)
.then(r => r.json())
.then(repos2 => {
// Handle the second page
})
}
})
}
// Usage
recursiveFetch('https://api.github.com/users/sindresorhus/repos', 100)
.then(repos => console.log(repos))
We need to return an array of repositories. This means we need to concatenate repos2 to repos before we can return repos.
const recursiveFetch = (link, perPage) => {
return fetch(`${link}?per_page=${perPage}`)
.then(r => r.json())
.then(repos => {
if (repos.length < perPage) {
return repos
} else {
fetch(`${link}?per_page=${perPage}&page=2`)
.then(r => r.json())
.then(repos2 => {
repos.concat(repos2)
})
}
})
}
// Usage
recursiveFetch('https://api.github.com/users/sindresorhus/repos', 100)
.then(repos => console.log(repos))
If repos2 has less than 100 items, we know we’re done. We can return the array of repositories we created. If repos2 has more than 100 items, we know we need to make another fetch request.
const recursiveFetch = (link, perPage) => {
return fetch(`${link}?per_page=${perPage}`)
.then(r => r.json())
.then(repos => {
if (repos.length < perPage) {
return repos
} else {
fetch(`${link}?per_page=${perPage}&page=2`)
.then(r => r.json())
.then(repos2 => {
repos.concat(repos2)
if (repos2.length < perPage) {
return repos
} else {
// Make another fetch request
}
})
}
})
}
// Usage
recursiveFetch('https://api.github.com/users/sindresorhus/repos', 100)
.then(repos => console.log(repos))
We need to make fetch requests until the server gives us less than 100 items. Here, we can call recursiveFetch again. This makes our function a recursive function.
const recursiveFetch = (link, perPage) => {
return fetch(`${link}?per_page=${perPage}`)
.then(r => r.json())
.then(repos => {
if (repos.length < perPage) {
return repos
} else {
// Note: recursiveFetch is not done yet!
return recursiveFetch(link, perPage)
}
})
}
// Usage
recursiveFetch('https://api.github.com/users/sindresorhus/repos', 100)
.then(repos => console.log(repos))
recursiveFetch is not done yet. We removed two important variables when we swapped the second fetch request with recursiveFetch.
We need a page number to make sure we fetch the correct page. Since we always begin recursiveFetch with the first page, we can default page to 1.
We increase page by 1 each time we call recursiveFetch.
const recursiveFetch = (link, perPage, page = 1) => {
return fetch(`${link}?per_page=${perPage}&page=${page}`)
.then(r => r.json())
.then(repos => {
if (repos.length < perPage) {
return repos
} else {
// Note: recursiveFetch is not done yet!
return recursiveFetch(link, perPage, page + 1)
}
})
}
// Usage
recursiveFetch('https://api.github.com/users/sindresorhus/repos', 100)
.then(repos => console.log(repos))
recursiveFetch returns the last page of items now. That’s not what we want. We want a list of all repositories Sindre has.
We need to create another parameter that stores the fetched repositories. Let’s call it items.
When the server sends us a response, we want to concatenate the results to items. Let’s call the results requestedItems instead of repos. This makes the variables clearer.
const recursiveFetch = (link, perPage, page = 1, items = []) => {
return fetch(`${link}?per_page=${perPage}&page=${page}`)
.then(r => r.json())
.then(requestedItems => {
items = items.concat(requestedItems)
if (requestedItems.length < perPage) {
return items
} else {
return recursiveFetch(link, perPage, page + 1, items)
}
})
}
// Usage
recursiveFetch('https://api.github.com/users/sindresorhus/repos', 100)
.then(repos => console.log(repos))
recursiveFetch works now.
Users can pass in four arguments to recursiveFetch. This makes the function confusing. We can make users pass in an object instead so each argument is clearly labeled.
const recursiveFetch = ({
link,
perPage,
page: 1,
items: []
}) => {
return fetch(`${link}?per_page=${perPage}&page=${page}`)
.then(r => r.json())
.then(requestedItems => {
items = items.concat(requestedItems)
if (requestedItems.length < perPage) {
return items
} else {
return recursiveFetch({
link,
perPage,
page: page + 1,
items
})
}
})
}
// Usage
recursiveFetch({
link: 'https://api.github.com/users/sindresorhus/repos',
perPage: 100
})
.then(repos => console.log(repos))
We can also use a ternary operator to make the if/else statement easier to read.
const recursiveFetch = ({
link,
perPage,
page: 1,
items: []
}) => {
return fetch(`${link}?per_page=${perPage}&page=${page}`)
.then(r => r.json())
.then(requestedItems => {
items = items.concat(requestedItems)
return requestedItems.length < perPage
? items
: recursiveFetch({
link,
perPage,
page: page + 1,
items
})
})
}
// Usage
recursiveFetch({
link: 'https://api.github.com/users/sindresorhus/repos',
perPage: 100
})
.then(repos => console.log(repos))
recursiveFetch functionrecursiveFetch function to fetch Sindre Shorhus’s repositoriesrecursiveXHR function to fetch Sindre Shorhus’s repositories 😎