POST /v1/scrape — Reference

Complete spec for the Web Unlocker endpoint. For usage examples and how-to explanations, see Web Unlocker.

POST https://api.omniscrape.io/v1/scrape

Headers required on every request:

Header	Value
`X-API-Key`	Your API key
`Content-Type`	`application/json`

Request body

Only url is required. All other fields default to the values shown.

Core

Field	Type	Default	Description
`url`	string	—	Required. Target URL including scheme (`https://`).
`method`	string	`GET`	HTTP method to use on the target: `GET` or `POST`.
`body`	object/string	—	Request body sent to the target when `method` is `POST`.
`mode`	string	`auto`	`auto`, `fast`, or `js_rendering`.
`output_format`	string	`html`	`html`, `markdown`, `plain_text`, `autoparse`, or `screenshot`.
`timeout`	integer	`30`	Max seconds to wait for the target (10–90).
`enable_solver`	boolean	`true`	Auto-solve anti-bot challenges.

Extraction

Field	Type	Default	Description
`css_selectors`	object	—	Map of key → CSS selector. Results in `data.css_extracted`.
`templates`	array	`[]`	Built-in extractors: `links`, `images`, `headings`, `emails`, `phone_numbers`, `metadata`, `tables`. Results in `data.template_extracted`.

JavaScript rendering

These fields force browser rendering even if mode is fast.

Field	Type	Default	Description
`js_wait_selector`	string	—	Wait until this CSS selector appears before returning.
`js_wait_timeout`	integer	`5000`	Max ms to wait for `js_wait_selector` (1000–60000).
`capture_xhr`	boolean	`false`	Capture background XHR/fetch calls. Results in `data.xhr_requests`.
`js_actions`	array	`[]`	Browser steps before returning. Forces `js_rendering`. Each step: `{ "action", "selector?", "value?" }`. Actions: `click`, `fill`, `select`, `check`, `uncheck`, `wait`, `wait_for`, `scroll_y`, `scroll_x`, `scroll_depth`, `load_more`, `next_page`.

Screenshots

Field	Type	Default	Description
`screenshot`	boolean	`false`	Return base64 PNG in `data.screenshot`.
`screenshot_type`	string	`viewport`	`viewport`, `fullpage`, or `element`.
`screenshot_selector`	string	—	CSS selector for `screenshot_type: element`.

Network & identity

Field	Type	Default	Description
`proxy`	string	—	Format: `residential:<cc>` or `residential:<cc>:<mode>`. Modes: `sticky` (default), `rotate`, `smart`.
`session_id`	string	—	Pin same IP and cookies across requests.
`custom_headers`	object	`{}`	Extra headers sent to the target.
`custom_cookies`	object	`{}`	Cookies sent with the request.

Unknown fields are silently ignored. Invalid values (e.g. bad mode) return 400.

Response body

{
  "success": true,
  "data": {
    "content": "<html>...</html>",
    "extracted_data": { "title": "Example" },
    "css_extracted": { "price": "$99.99" },
    "template_extracted": { "links": ["..."] },
    "xhr_requests": [{ "url": "...", "body": "..." }],
    "screenshot": "iVBORw0KGgo...",
    "status_code": 200,
    "final_url": "https://example.com",
    "headers": { "content-type": "text/html" },
    "cookies": { "session": "abc123" }
  },
  "metadata": {
    "method_used": "js_rendering",
    "output_format": "html",
    "elapsed_time": 5.23,
    "solver_used": true,
    "challenge_solved": true,
    "timestamp": "2026-06-23T10:30:00Z"
  },
  "billing": {
    "charged": 0.0035,
    "cost_per_request": 0.0035,
    "balance_after": 49.91
  }
}

`data` fields

Field	Present when	Description
`content`	Always (unless extractor-only)	Page body in the requested `output_format`.
`extracted_data`	`output_format: autoparse`	Auto-detected structured data.
`css_extracted`	`css_selectors` provided	Your mapped selector results.
`template_extracted`	`templates` provided	Built-in extractor results.
`xhr_requests`	`capture_xhr: true`	Background XHR/fetch payloads captured during render.
`screenshot`	`screenshot: true`	Base64-encoded PNG.
`status_code`	Always	HTTP status returned by the target.
`final_url`	Always	URL after redirects.
`headers`	Always	Response headers from the target.
`cookies`	Always	Cookies set by the target.

`metadata` fields

Field	Type	Description
`method_used`	string	Which lane ran: `fast` or `js_rendering`.
`output_format`	string	Format applied to `content`.
`elapsed_time`	float	Total processing time in seconds.
`solver_used`	boolean	Whether the anti-bot solver ran.
`challenge_solved`	boolean	Whether a challenge was successfully solved.
`timestamp`	string	ISO 8601 completion time.

`billing` fields

Field	Type	Description
`charged`	float	Amount billed (USD). `0` on failed unlocks.
`cost_per_request`	float	Standard price — `0.0035`.
`balance_after`	float	Remaining balance after the charge.

Error response

{
  "success": false,
  "error": "Target returned 403 after solver attempts",
  "code": "UNLOCK_FAILED",
  "metadata": { "elapsed_time": 12.4, "solver_used": true }
}

HTTP	When
`400`	Bad parameter. Fix the request body.
`401`	Missing or invalid `X-API-Key`.
`402`	Balance exhausted, trial expired, or quota exceeded. Check `code`.
`429`	Concurrency limit. Back off and retry.
`500/503/504`	Server error or timeout. Retry with backoff.
`502`	Unlock failed. Not billed.

See Errors for the full list of code values and retry strategy.

Async variant

To submit without holding a connection open:

POST /v1/scrape/async   →  { "job_id": "..." }
GET  /v1/jobs/{job_id}  →  { "status": "completed", "result": { ... } }

Same request body. See Async Jobs.

Request body​

Core​

Extraction​

JavaScript rendering​

Screenshots​

Network & identity​

Response body​

data fields​

metadata fields​

billing fields​

Error response​

Async variant​