JavaScript Rendering

Many modern sites render content with JavaScript after the initial HTML loads. To capture that content, OmniScrape runs the page in a real browser.

When the browser runs

A real browser is used in these cases:

1. mode: "js_rendering" — always uses a browser.
2. mode: "auto" — escalates to a browser automatically when the page returns an empty shell or a challenge.
3. Browser features requested — using js_wait_selector, capture_xhr, or screenshots forces browser rendering even if you picked fast.

So you rarely set a flag for "render JS" — you either pick js_rendering, or use a feature that needs it.

Waiting for content

The most common mistake is reading the page before its content appears. Use js_wait_selector to wait for a specific element. Pair it with js_wait_timeout (milliseconds) to cap how long to wait.

{
  "url": "https://app.example.com/results",
  "mode": "js_rendering",
  "js_wait_selector": ".results-list .item",
  "js_wait_timeout": 8000
}

If the selector appears in 400 ms, the request returns immediately; if it never appears, the request returns after js_wait_timeout ms. The default timeout is 5000 ms.

Capturing background API calls

Single-page apps often fetch their data from a JSON API after load. Instead of scraping the rendered DOM, you can capture those XHR/fetch responses directly with capture_xhr.

{
  "url": "https://app.example.com/products",
  "mode": "js_rendering",
  "capture_xhr": true,
  "js_wait_selector": ".product-card"
}

The captured calls are returned under data.xhr_requests. This is often the cleanest way to get structured data — you read the site's own API response rather than parsing HTML.

Combining with extraction

JS rendering pairs naturally with CSS extraction. Wait for the element, then extract it:

{
  "url": "https://shop.example.com/p/1",
  "mode": "js_rendering",
  "js_wait_selector": ".price-amount",
  "css_selectors": { "price": ".price-amount" }
}

When you need full interaction

Rendering waits for content but doesn't click buttons, fill forms, or log in. For interactive, multi-step flows, use Browser-as-a-Service.