Rate Limits & Concurrency

OmniScrape limits how many requests you can have in flight at the same time (concurrency), based on your plan. There is no fixed requests-per-second cap — you can send as many requests as you like as long as you stay under your concurrency limit.

Concurrency by plan

Plan	Max concurrent requests
Free trial	1
Pay-As-You-Go	5
Startup	10
Growth	25

A "concurrent request" is any scrape (sync or async) that is currently being processed. When a request finishes, its slot frees up immediately.

What happens at the limit

If you exceed your concurrency limit, the request returns 429 with a structured code and a Retry-After header:

{
  "success": false,
  "error": "Concurrent request limit reached for your plan. Retry shortly.",
  "code": "CONCURRENCY_LIMIT"
}

Wait the suggested time (or a couple of seconds) and retry.

Monthly quotas

Subscription plans include a monthly allowance of successful requests. Pay-As-You-Go has no monthly cap — you simply pay per request from your balance.

Plan	Included successful requests / month
Pay-As-You-Go	Unlimited (balance-based)
Startup	10,000
Growth	40,000

Exceeding the monthly quota returns 402:

{
  "success": false,
  "error": "monthly quota exceeded: 10000/10000 requests used this month",
  "code": "MONTHLY_QUOTA_EXCEEDED"
}

Upgrade your plan or wait for the monthly reset. See Pricing.

Staying within limits

The simplest pattern is a bounded worker pool sized to your plan's concurrency:

import os, requests
from concurrent.futures import ThreadPoolExecutor

KEY = os.environ["OMNISCRAPE_KEY"]
MAX_CONCURRENCY = 5  # match your plan

def scrape(url):
    return requests.post(
        "https://api.omniscrape.io/v1/scrape",
        headers={"X-API-Key": KEY},
        json={"url": url, "mode": "auto"},
        timeout=120,
    ).json()

urls = [f"https://example.com/page/{i}" for i in range(1, 101)]
with ThreadPoolExecutor(max_workers=MAX_CONCURRENCY) as pool:
    results = list(pool.map(scrape, urls))

For very large batches, prefer the async jobs API so you submit work without holding connections open, and still cap how many jobs are unresolved at once.

Retry guidance

Retry 429, 500, 502, and 503 with exponential backoff. Do not retry 400, 401, or 402. See Errors for the full strategy.