API Rate Limits in n8n: Pace, Back Off, Then Shed Load

Handle API rate limits in n8n by classifying the call, pacing the happy path, then backing off with the vendor’s Retry-After (or documented cool-down) — not by hammering Retry On Fail until the vendor locks you out.

Unlimited retry is how a single burst becomes a multi-hour outage. This post is the production framing we use at Spurlock Studios. It sits inside the Production n8n handbook.

The short answer

• 429 means stop and wait — the vendor is throttling you. Read Retry-After when present; do not invent a random backoff that undercuts their cool-down.
• Retry On Fail is not a rate limiter — it is for transient blips. Bound it. Pair it with Wait / batch pacing on known-slow APIs.
• Pace before you retry — Split In Batches + Wait keeps you under the cap so you never enter the 429 spiral.
• Shed noncritical work — enrichment can fail closed or skip; CRM writes and money moves cannot thrash the same budget.
• Webhook retries multiply load — a slow 429 loop plus provider redelivery is a stampede. Cap concurrency and use idempotency.

What HTTP 429 actually means for your design

Signal	Meaning	Design response
429 Too Many Requests	You exceeded a rate or quota window	Pause; honor cool-down; reduce concurrency
Retry-After: N (seconds) or HTTP-date	Vendor-stated wait	Wait at least that long before the next attempt
No Retry-After	Cool-down is undocumented or vendor-specific	Use the vendor’s published wait (example below) or a conservative bound
5xx with retry noise	Often transient infra, not your rate budget	Separate retry class from 429

Treat 429 as backpressure, not as “try harder.” Trying harder is how you burn the next window too.

When Retry On Fail is enough

Use node-level Retry On Fail when:

1. Failures are rare and short (one blip, not sustained throttle).
2. The node is not inside a hot fan-out (hundreds of parallel HTTP calls).
3. You set a max tries and a delay that matches the vendor, not “infinite.”
4. Side effects are idempotent so a late success cannot double-write.

When any of those fail, you need pacing or an external queue — not more retries.

When you need Split In Batches + Wait

Pace the happy path for APIs with hard per-second caps or shared budgets across workflows.

Minimum pattern:

1. Collect items (or receive a list).
2. Split In Batches — batch size sized to the vendor cap and your parallel branches.
3. Wait between batches — long enough that your peak req/s stays under the limit.
4. Run the HTTP / vendor node on the batch.
5. On 429: Wait using Retry-After (or vendor cool-down), then retry that batch once with a bound.

Example Wait expression when the previous HTTP node exposed headers (map the header into the item first):

// Seconds from Retry-After, floor at vendor minimum if header missing
const h = $json.headers?.["retry-after"] ?? $json.headers?.["Retry-After"];
const sec = Number(h);
return Number.isFinite(sec) && sec > 0 ? sec : 30;

Guessing Math.random() * 5 while the vendor wants thirty seconds is how you stay rate-limited.

Honor Retry-After instead of guessing

Decision list for every production HTTP path:

1. Capture response status and headers on failure (Error Trigger / Continue On Fail with branch).
2. If status is 429 and Retry-After is a number → Wait that many seconds.
3. If Retry-After is an HTTP-date → Wait until that time (or skip and alert if too far out).
4. If no header → use the vendor-documented cool-down, not folklore.
5. Retry once (or a small bound). Then stop and DLQ or shed.

Airtable’s Web API documents a concrete case: 5 requests per second per base, plus 50 requests per second across all traffic for a personal access token or service account. Exceeding those returns 429, and Airtable states you must wait 30 seconds before subsequent requests succeed (Airtable rate limits). Their error docs repeat the same 30-second cool-down (Airtable errors). Pace to stay under 5/s; on 429, wait at least 30s — do not chip away with 2-second retries.

Classify critical vs enrichment before you share a budget

Work class	Example	On sustained 429
Critical write	Create deal, post invoice, route lead	Bound retry → DLQ → human; pause noncritical siblings
Critical read that gates a write	Fetch account before update	Same as write — do not invent the row
Enrichment	Firmographics, AI summary, nice-to-have fields	Fail closed (skip field) or fail open only if product accepts unknown
Bulk backfill	Nightly sync	Lower concurrency; extend window; never share burst budget with webhooks

If enrichment and lead routing share one Airtable base and one token, enrichment will starve routing during a scrape. Separate credentials/bases when the business requires it, or shed enrichment first.

Failure mode: unlimited retry + webhook redelivery

What breaks: a webhook fires; your HTTP node hits 429; Retry On Fail loops; the provider times out and redelivers; now you have N executions all retrying the same throttle.

What it costs: hours of CRM quiet, duplicate side effects if anything eventually succeeds without idempotency, and a muted Slack channel full of identical errors.

What you do instead:

• Cap retries (small N).
• Honor cool-down.
• Cap workflow concurrency / use a queue for bulk.
• Idempotency key before irreversible nodes.
• Alert once with execution deep link, not once per retry.

Rate-limit across multiple workflows

n8n does not give you a global “Airtable 5/s” governor out of the box. Shared budget patterns:

Pattern	When	Tradeoff
One “API gateway” workflow	Many callers, one vendor	Extra hop; clear ownership
External queue (Redis / SQS) + single consumer	High fan-in	Real ops; true serialization
Stagger schedules	Cron-heavy estates	Easy; weak under webhook bursts
Separate tokens / bases	Hard isolation needed	Cost and admin overhead

Checklist for a shared base:

• Inventory every workflow that hits the vendor
• Tag critical vs enrichment
• Cap concurrent executions on the hot paths
• Put bulk jobs on off-peak schedules
• One alert owner for 429 storms

Bursts from webhook retries

Providers deliver at least once. Slow handlers invite redelivery. Rate limits make handlers slower. That feedback loop is the stampede.

Controls that work together:

1. Fast ack path where the vendor allows (queue the work, respond 200).
2. Idempotency before writes.
3. Bounded concurrency on the consumer.
4. Separate enrichment so it cannot block the ack path.

If you cannot ack fast, your Wait nodes must still honor vendor cool-downs — and your DLQ must catch poison after the retry budget.

When you need an external queue

Stay inside n8n pacing when volume is moderate and one or two workflows own the vendor.

Add an external queue when:

• Many producers share one low cap (classic Airtable/base case).
• You need fair scheduling across clients or brands.
• Backfills must not starve interactive webhooks.
• You already run Redis/SQS for other reasons and can put a single consumer in front of the API.

Queue mode in n8n (workers + Redis) scales execution concurrency — it does not replace a per-vendor rate governor. Different problem. See the handbook spine for how pacing sits next to DLQ and schema checks.

Size the batch to the cap (worked example)

Airtable at 5 req/s per base (docs):

Design choice	Example setting	Why
Batch size	5 items if each item = 1 request	Stays at the ceiling only if Wait is ≥1s
Wait between batches	≥1 second (prefer 1.1–1.2s)	Leaves headroom for other workflows on the same base
Parallel branches	1 HTTP lane on that base	Two parallel 5/s lanes are 10/s — instant 429
On 429	Wait ≥30s, then one bounded retry	Matches Airtable’s published cool-down

If three workflows share the base, pretend you have ~1–2 req/s each until you measure. Shared fiction beats shared outage.

HTTP Request node settings that matter

For the n8n HTTP Request node on throttled vendors:

1. Retry On Fail — on, but max tries small (2–3).
2. Wait Between Tries — at least the vendor cool-down when you know it; otherwise read Retry-After in an error branch instead of a fixed undersized delay.
3. Timeout — long enough for the vendor, short enough that webhook providers do not stack redeliveries forever.
4. Continue On Fail — only when you have an explicit IF on $json / error status next; never to swallow 429 into a fake success.

# Example HTTP Request options (UI equivalents)
Retry On Fail: true
Max Tries: 3
Wait Between Tries: 30000   # ms — only if vendor cool-down is 30s and header absent

Prefer header-driven Wait over a hardcoded 30s when the API sends Retry-After.

Monitor 429 before customers do

Signal	Where	Action threshold
Count of 429 responses / hour	Error workflow → metrics or log drain	Page if above baseline ×3
Mean Wait time inserted	Custom metric or execution notes	Rising Wait = budget pressure
DLQ age for errorClass=rate_limit	DLQ table	Items older than SLA → shed enrichment
Webhook redelivery rate	Provider dashboard	Climbing with your latency → cut work on the hot path

One Slack message with deep links beats fifty identical “Rate limit exceeded” lines.

Shed load procedure (when the storm is already on)

1. Pause enrichment and bulk sync workflows sharing the token/base.
2. Leave critical write path running at reduced concurrency.
3. Drain in-flight retries; do not Replay All.
4. Confirm 429 rate drops.
5. Resume enrichment at half prior batch size.
6. Schedule the architecture fix (gateway consumer or separate base) within a week.

Shedding is not permanent architecture. It is how you buy the hour to fix architecture.

Operator checklist (ship this week)

• Every production HTTP node: max retries set, not unlimited
• 429 path reads Retry-After or vendor cool-down
• Hot lists use Split In Batches + Wait sized to the cap
• Enrichment can be skipped without failing the critical path
• Shared-token inventory exists
• 429 storm alert pages a human once, with a mute plan
• Idempotency on irreversible side effects
• Shed procedure written where on-call can find it

FAQ

Why does unlimited retry make rate limits worse?

Each failed attempt still counts against many vendors’ windows, and aggressive retries keep you pinned at the ceiling. You never drain the cool-down, so every subsequent call fails too. Bound retries and wait the documented interval.

How do I rate-limit across multiple workflows?

n8n will not automatically share a per-base budget across workflows. Inventory callers, pace each hot path, stagger bulk jobs, and for hard caps put a single consumer (gateway workflow or external queue) in front of the API.

What about Airtable’s 5 requests/second?

Airtable’s Web API is limited to 5 requests per second per base, and 50 requests per second for all traffic using a given personal access token or service account. On exceed you get 429 and must wait 30 seconds before requests succeed again (docs). Design batches under 5/s; on 429 wait ≥30s.

Should enrichment fail open or closed?

Default fail closed: skip the enrichment field and continue the critical write when the product accepts a thinner record. Fail open (proceed without data) only when a missing enrichment cannot corrupt downstream decisions. Never let enrichment retries starve critical writes on the same budget.

How do bursts from webhook retries interact with limits?

Provider redelivery plus your own Retry On Fail multiplies concurrent calls into the same throttle. Cap retries, ack or queue quickly, use idempotency, and keep bulk/enrichment off the webhook hot path.

When do I need an external queue?

When many producers share a low vendor cap, when backfills must not starve interactive traffic, or when you need fair multi-tenant scheduling. n8n Wait/batch is enough for a single paced workflow; shared estates need a real governor.

CTA

Pace first, honor the cool-down, then shed — retries are the last tool, not the first.

For the full production spine, keep the handbook open. When you want a rate-limit and backpressure review on your stack, use automation or book a call.