Skip to main content

Sessions

A session keeps the same IP address and cookie jar across multiple requests. Use one whenever a sequence of requests needs to look like a single, continuous visitor — paginated crawls, multi-step flows, or anything behind a login.

Creating a session​

Pick any string as a session_id and reuse it. The first request establishes the session; later requests with the same id reuse its IP and cookies.

{
  "url": "https://example.com/page/1",
  "session_id": "crawl-2026-06-23"
}
{
  "url": "https://example.com/page/2",
  "session_id": "crawl-2026-06-23"
}

Both requests share one IP and accumulate cookies, so pagination and "remember me" state work as expected.

When to use a session​

  • Pagination — sites that tie page state to a cookie or IP.
  • Logged-in scraping — authenticate once, reuse the cookie.
  • Multi-step flows — add to cart, then check out.
  • Rate-sensitive sites — a stable identity looks more human than a new IP each time.

When NOT to use a session​

For independent, parallel requests across many unrelated URLs, omit session_id. Fresh IPs spread load and reduce the chance of one IP getting flagged.

Lifetime​

Sessions stay alive while you keep using them and expire after roughly 10 minutes of inactivity. After expiry, the same session_id starts a new session with a new IP. For long crawls, keep requests flowing or accept that a new IP may be assigned.

Combining with geo-targeting​

Pin both a country and a session to crawl a regional site as one consistent visitor:

{
  "url": "https://shop.example.com/de/page/1",
  "proxy": "residential:de",
  "session_id": "de-shop-crawl"
}