Skip to main content
If many requests fail at once, check the status page first. Start with the response. Every error carries an error_code and a retryable flag, and every response carries meta.requestId. Quote the requestId if you contact support.

​
Authentication

​
Too many requests

See Rate limits.

​
A page comes back empty or incomplete

  • The content loads late. Add wait_for (milliseconds) or wait_for_selector so the page finishes rendering first.
  • The main content was hard to find. A warnings entry says the full page body was returned instead. Name the content with include_selectors, or drop clutter with exclude_selectors.
  • The site blocks bots (TARGET_BLOCKED). Retrying won’t help. Some sites refuse every automated request.
  • It’s a file, not a page (TARGET_NOT_A_PAGE). PDFs are read as documents. Other downloads aren’t.
  • summary or json is null. The language model step failed on its own. The reason is in warnings, and the rest of the scrape is still there.
  • A json field is null. The value isn’t on the page. Prefetch never fills fields in from a model’s own knowledge.

​
A crawl returns fewer pages than expected

Call GET /crawl/{id}/errors. It counts every skipped URL by reason: The page links to none of the site’s own pages, and no sitemap lists any. This is common on apps that build their links with JavaScript after user input. Try sitemap=only, or map a different page on the site.

​
Webhooks don’t arrive

  • The URL must be public and use https://. localhost never works. Use a tunnel while you develop.
  • Answer with a 2xx within 10 seconds, and do slow work after you respond.
  • Webhooks aren’t available through RapidAPI keys.
  • Missed one? The status endpoint always has every result. See Webhooks.

​
A monitor is noisy or quiet

​
Logos show a letter mark

The logo CDN shows a letter mark while it looks a new domain’s logo up in the background. The next request usually has the real logo. Use fallback=404 or fallback=transparent to handle it yourself.

​
Still stuck?

Email [email protected] with the requestId, the request you sent, and what you expected.