Why cursors at all
Offset/limit pagination is intuitive and broken. The moment a row is inserted at the top of the dataset, every offset shifts by one. Your next page starts with the row your previous page ended with, and the row that should have been first on the new page is silently skipped. For a catalog that adds tracks daily, that's a guaranteed source of double-counts and missing rows.
Cursor pagination encodes the position itself in an opaque token. The cursor returned with page N is the exact instruction for “start from where N ended”, and it stays stable regardless of inserts above it. Every paginated v1 endpoint uses this pattern:
{
"results": [
{ "id": 789, "title": "Resonance EP", /* … */ },
/* … */
],
"next_cursor": "2026-03-22:78:789"
}The last page returns next_cursor: null:
{
"results": [
{ "id": 42, /* last few rows */ }
],
"next_cursor": null
}next_cursor in the response and the query parameter is cursor. Never nextCursor, pageToken, or anything else. Same shape on every paginated endpoint.The basic walker
The minimum shape: loop, attach cursor when present, stop when next_cursor is null:
async function walkAll<T>(url: string): Promise<T[]> { const out: T[] = []; let cursor: string | null = null; while (true) { const u = new URL(url); if (cursor) u.searchParams.set("cursor", cursor); const res = await fetch(u, { headers: { "x-api-key": API_KEY } }); if (!res.ok) throw new Error(`${res.status} ${res.statusText}`); const page = await res.json(); out.push(...page.results); if (!page.next_cursor) break; cursor = page.next_cursor; } return out; } const all = await walkAll<Release>(`${BASE}/labels/1/releases?limit=100`); console.log(`Pulled ${all.length} releases`);
Generic over the row type, takes any URL that returns a cursor-paginated response. Use it anywhere: search, releases, suggestions, label feeds.
Make the walk resumable
For long walks (anything that takes more than a few minutes, anything in a cron job), persist the cursor to disk after every page. A crash or sigterm doesn't restart from page 1:
import fs from "node:fs/promises"; // Resumable walker - survives crashes / sigterm. Cursor goes to disk after // every page, so a restart picks up from the last successfully-processed page. async function resumableWalk<T>( url: string, cursorFile: string, onPage: (rows: T[]) => Promise<void>, ) { let cursor: string | null = null; try { cursor = (await fs.readFile(cursorFile, "utf-8")).trim() || null; } catch { /* first run */ } while (true) { const u = new URL(url); if (cursor) u.searchParams.set("cursor", cursor); const res = await fetch(u, { headers: { "x-api-key": API_KEY } }); const page = await res.json(); await onPage(page.results); // 1. do the work await fs.writeFile(cursorFile, page.next_cursor ?? ""); // 2. persist cursor if (!page.next_cursor) break; cursor = page.next_cursor; } } await resumableWalk<Release>( `${BASE}/labels/1/releases?limit=100`, "./.state/labels-1.cursor", async (rows) => { for (const r of rows) await indexRelease(r); }, );
Don't try to parallelise a single walk
You can't, because each cursor only comes from the previous response. The trap is to fetch page 1 in serial, then try to issue 10 parallel requests for pages 2–11. Page 2's cursor doesn't exist until page 1 returns, and pages 3–11 don't exist until each predecessor returns. There's no way to skip ahead.
The right kind of parallelism is across independent walks: one walker per label, one per artist, one per search query. Each walker is sequential internally; the parallelism is over the set of walkers:
// Right: parallelise across independent walks (e.g. per label). const labelIds = [1, 42, 128, 256]; const allByLabel = await Promise.all( labelIds.map(id => walkAll<Release>(`${BASE}/labels/${id}/releases?limit=100`)), ); console.log(`Pulled ${allByLabel.flat().length} releases across ${labelIds.length} labels`);
Watch for max-page-size limits
Most paginated endpoints accept limit between 1 and 100. /v1/tracks/browseis an outlier: it returns at most 20 results and has no cursor at all (it's designed for discovery, not enumeration). If you need to enumerate tracks for a label or artist, use /v1/labels/:id/releases or /v1/artists/:id/releases and walk through them. Those carry tracks via release detail.
Going further
- Idempotency at the consumer. Even with stable cursors, network retries can cause the same page to be processed twice. Make your
onPageupserts idempotent, keyed by row ID, so reprocessing is harmless. - Backoff on 429. Rate limits return
429with aRetry-Afterheader. Sleep that many seconds and resume; your cursor is still valid. - Cap walks with a date cutoff. For
/v1/releases/newand other date-ordered endpoints, you rarely want to walk all the way to the start of the catalog. Stop whenrelease_datedrops below your floor.
Frequently asked questions
Why cursors instead of offset/limit?
Offsets break the moment the dataset changes mid-walk. A new release added at the top shifts every offset by one, so page 2 starts with the row page 1 ended with. Cursor pagination encodes a stable position so the next page picks up exactly where the previous one left off, regardless of inserts.
Can I parallelise pagination?
Not naively. Each cursor only comes from the previous response. You can speed up by fetching multiple disjoint windows in parallel (e.g. one walker per label ID), but you can't parallelise a single walk. If sequential is too slow, the better fix is usually a different endpoint (browse with filters) rather than more concurrency.
Does the cursor expire?
No. Cursors encode a position, not a session. A cursor from yesterday's response still works tomorrow. The only thing that “expires” is the data: if you resume a walk after 24 hours, the rows you're paginating over may have changed, but the cursor itself still points to the right place.
What if next_cursor is null but I expected more results?
nullmeans “no more results”, so you've reached the end. If the total is smaller than expected, double-check your filters (especially for /v1/tracks/browse, where genre/year may be more restrictive than you think). Some endpoints also cap at a hard limit (e.g. /v1/tracks/browse returns at most 20).