Skip to main content
A batch runs one endpoint over a list of URLs, up to 500, as a single background job. You send one request instead of hundreds, and Prefetch manages concurrency for you. Each URL is answered exactly as a single call would answer it: curated data first, then a live read. A batch scrape fetches only the URLs you list. It follows no links and reads no sitemap. To discover pages, use a crawl.

Example requests

Example response

Batch ids start with crw_, like crawl ids, because batches run on the same job system. Read results with GET /batch/{endpoint}/{id}, for example GET /batch/enrich/crw_7c2e…. See Async jobs for polling, pagination and expiry.

Results

Each result carries every field the single endpoint returns for that URL (domain, brand, company, classification, screenshot_url, styleguide, markdown…), plus: GET /batch/{endpoint}/{id}/errors lists only the URLs that failed, so you can resubmit the retryable ones.

Options

Endpoint options

For /batch/{endpoint}, options are that endpoint’s own parameters, applied to every URL. Both snake_case and camelCase work, as typed JSON or as strings: For /batch/scrape, use scrape_options instead. It takes everything /scrape takes. Options are checked before anything is queued. An invalid option is a 400 that names it.

Job options

Duplicates, including the same site under another spelling (www., a trailing slash), run once.

Webhooks

webhook works as it does for crawls. Events are named for the endpoint: batch_enrich.started, batch_enrich.page (one per URL, carrying its result), batch_enrich.completed and batch_enrich.failed, or batch_scrape.* for a batch scrape. Each delivery is signed with the secret returned when the batch is created. See Webhooks. Webhooks aren’t available through RapidAPI. Poll the status endpoint instead.

Credits

Submitting is free. Each URL answered costs what one call of that endpoint costs, charged as URLs complete, and never twice. Failed URLs are free. A /batch/{endpoint} job your key can’t pay for in full is refused up front with CREDITS_EXHAUSTED, and the message says how many URLs fit. A 500-domain /batch/enrich can cost up to 6,000 credits.

Over MCP

prefetch_batch and prefetch_batch_scrape submit a batch and wait for it, returning the results if it finishes in time and an id for prefetch_check_crawl_status if not.

Next

Build a company directory

A recipe that enriches hundreds of domains with one batch.

API reference

Every parameter, and a playground to try it.