Rate limits and retries

Rate limits#

Limits are per key and per endpoint, not per account. A separate key per environment gets you separate budgets. Each endpoint has its own budget. A route's original name and its new name are one endpoint, so they share one budget.

Each ingest request carries up to 500 works, so 60 requests per minute is up to 30,000 works per minute.

A partner can agree a different limit on an endpoint with us. That limit replaces the one in the table, for the keys of that partner only. Ask us to confirm the limit on your keys.

A request over the limit returns 429 RATE_LIMITED with a Retry-After header. The value is a whole number of seconds. Wait that long, then retry the same request. Do not retry sooner.

HTTP/1.1 429 Too Many Requests
Retry-After: 12
x-request-id: 3f1c2a9e-6b7d-4c2e-9a1f-0d5e8b7c6a54

Idempotency#

A commit accepts an Idempotency-Key header, so a network failure never causes a double write. Dry-runs ignore the header, because they store nothing.

Send a unique key per logical request. A UUID works well. When you page a large catalogue, use one key per chunk.

We remember keys for 24 hours.

The durable guarantee#

An Idempotency-Key protects a retry for 24 hours. A stable client_work_ref protects it forever.

client_work_ref is the upsert key. Re-sending a work with the same ref updates that work in the database rather than creating a second one. A retry outside the idempotency window still cannot create a duplicate.

Use both. The Idempotency-Key is the convenience layer. The client_work_ref is the durable one.

A retry loop#

key = uuid()
backoff = 1s

while true:
    res = POST /api/v1/catalogue/works
          Authorization: Bearer trk_live_...
          Idempotency-Key: key
          body: { client_artist_ref, catalog_name, works: chunk }

    if res.status in (200, 201):
        break                              # success; an idempotent replay counts as success

    if res.status == 429:
        sleep(res.headers["Retry-After"])  # honour the header, do not guess
        continue

    if res.status in (500, 502, 503, 504):
        sleep(backoff)
        backoff = min(backoff * 2, 60s)
        continue

    raise res                              # any other 4xx: correct the request, do not retry