> ## Documentation Index
> Fetch the complete documentation index at: https://docs.prefetch.io/llms.txt
> Use this file to discover all available pages before exploring further.
# GET /map
> Discover every URL a site exposes, from its sitemaps and on-page links, without fetching them. Costs 3 credits.
## Overview
`/map` answers "what pages does this site have?" without downloading any of them. It merges two sources:
* **Sitemaps** — found through `robots.txt` first, then the conventional locations (`/sitemap.xml`, `/sitemap_index.xml`). Sitemap indexes are followed one level deep. Authoritative and cheap: one request can describe a whole site.
* **On-page links** — every link on the URL you pass. The fallback for sites that publish no sitemap, and the only source that reflects what is actually linked.
A URL found in both appears once. Sitemap provenance wins, because it carries `lastmod` and is the site's own statement about its content, but the anchor text from the page is kept as a title.
Use it to plan a crawl, to find one page without guessing its path, or to check what a site has published recently.
## Example requests
```bash Everything theme={null}
curl "https://api.prefetch.io/map?url=https://stripe.com" \
-H "X-API-Key: $PREFETCH_API_KEY"
```
```bash Search theme={null}
curl "https://api.prefetch.io/map?url=https://stripe.com&search=pricing&limit=20" \
-H "X-API-Key: $PREFETCH_API_KEY"
```
```bash Sitemaps only theme={null}
curl "https://api.prefetch.io/map?url=https://stripe.com&sitemap=only" \
-H "X-API-Key: $PREFETCH_API_KEY"
```
```javascript Node.js theme={null}
const params = new URLSearchParams({
url: "https://stripe.com",
search: "pricing",
limit: "20",
});
const res = await fetch(`https://api.prefetch.io/map?${params}`, {
headers: { "X-API-Key": process.env.PREFETCH_API_KEY },
});
const { data } = await res.json();
console.log(`${data.count} of ${data.total_discovered} links`);
data.links.forEach((l) => console.log(l.url));
```
```python Python theme={null}
import requests, os
r = requests.get(
"https://api.prefetch.io/map",
params={"url": "https://stripe.com", "search": "pricing", "limit": 20},
headers={"X-API-Key": os.environ["PREFETCH_API_KEY"]},
)
for link in r.json()["data"]["links"]:
print(link["url"], link["source"])
```
## Example response
```json theme={null}
{
"success": true,
"data": {
"url": "https://stripe.com",
"domain": "stripe.com",
"links": [
{
"url": "https://stripe.com/pricing",
"title": "Pricing",
"description": null,
"lastmod": "2026-07-02",
"source": "sitemap"
},
{
"url": "https://stripe.com/enterprise/pricing",
"title": null,
"description": null,
"lastmod": null,
"source": "sitemap"
}
],
"count": 2,
"total_discovered": 2,
"sources": { "sitemap": 2, "links": 0 },
"sitemaps_used": ["https://stripe.com/sitemap/sitemap.xml"]
},
"meta": {
"requestId": "b4e3d2c1-8a7b-4c3d-9e2f-1a3b5c7d9e1f",
"durationMs": 1830
}
}
```
## Reading the response
| Field | Meaning |
| ---------------------- | ----------------------------------------------------------------------------------------------------------- |
| `count` | Links returned, after `limit` was applied. |
| `total_discovered` | Links found before `limit`. A much larger number means you are seeing a slice. |
| `sources` | How many returned links came from each source. `{"sitemap": 0}` means the site publishes no usable sitemap. |
| `sitemaps_used` | The sitemap documents that actually produced entries. |
| `source` on each link | `sitemap` or `links`. |
| `lastmod` on each link | Present only for sitemap entries whose sitemap declared one. |
## Ordering and search
Without `search`, links come back **shortest path first** — so the homepage and top-level sections lead, and the deep long tail follows.
With `search`, results are **filtered as well as ranked**: links matching nothing are dropped, not just pushed down. A match in the URL path outranks one in the link text.
```bash theme={null}
# Returns only pricing-related URLs
curl "https://api.prefetch.io/map?url=https://stripe.com&search=pricing" \
-H "X-API-Key: $PREFETCH_API_KEY"
```
## Choosing sources
| `sitemap` | Behavior |
| --------- | -------------------------------------------------------------------------------------------------- |
| `include` | Default. Fetches the page **and** the sitemaps, then merges. |
| `only` | Sitemaps only. Skips the page fetch, so it is faster and works even when the homepage blocks bots. |
| `skip` | On-page links only. Use when a site's sitemap is stale and you want what is actually linked today. |
`robots.txt` is read for its `Sitemap:` entries regardless. `/map` fetches at most one page of the site, so nothing here is affected by `Disallow` rules.
## Scope
By default the result stays on the site you asked about, subdomains included, and URLs that differ only by query string collapse into one.
* `include_subdomains=false` — restrict to the exact host. `blog.stripe.com` is excluded when you map `stripe.com`.
* `ignore_query_params=false` — keep `?page=2` and `?page=3` as separate entries.
Links to other sites are never returned.
## Related
Fetch the content of any URL you found. **3 credits.**
Fetch all of them in one job. **3 credits + 3 per page.**
## OpenAPI
````yaml GET /map
openapi: 3.1.0
info:
title: Prefetch API
description: >-
Extract brand identity, company data, screenshots, IAB classifications, and
clean page content from any URL.
version: 1.0.0
contact:
email: support@prefetch.io
servers:
- url: https://api.prefetch.io
description: Production
security:
- apiKey: []
tags:
- name: Endpoints
description: Data extraction endpoints. All require authentication and consume credits.
- name: Health
description: Health and readiness probes. No authentication required.
paths:
/map:
get:
tags:
- Endpoints
summary: Map a site's URLs
description: >-
Discovers the URLs a site exposes without fetching them: its sitemaps,
found through `robots.txt` and the conventional locations, merged with
the links on the page itself. **Credit cost: 3.**
operationId: mapSite
parameters:
- $ref: '#/components/parameters/url'
- $ref: '#/components/parameters/mapLimit'
- $ref: '#/components/parameters/mapSearch'
- $ref: '#/components/parameters/mapSitemap'
- $ref: '#/components/parameters/mapIncludeSubdomains'
- $ref: '#/components/parameters/mapIgnoreQueryParams'
responses:
'200':
description: Links discovered.
content:
application/json:
schema:
type: object
properties:
success:
type: boolean
enum:
- true
data:
$ref: '#/components/schemas/MapData'
meta:
$ref: '#/components/schemas/Meta'
required:
- success
- data
- meta
example:
success: true
data:
url: https://stripe.com
domain: stripe.com
links:
- url: https://stripe.com/pricing
title: Pricing
description: null
lastmod: '2026-07-02'
source: sitemap
- url: https://stripe.com/enterprise/pricing
title: null
description: null
lastmod: null
source: sitemap
count: 2
total_discovered: 2
sources:
sitemap: 2
links: 0
sitemaps_used:
- https://stripe.com/sitemap/sitemap.xml
meta:
requestId: b4e3d2c1-8a7b-4c3d-9e2f-1a3b5c7d9e1f
durationMs: 1830
'400':
description: Invalid or missing parameters.
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
'401':
description: Missing API key.
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
'403':
description: Revoked key, expired key, credit limit, or blocklisted URL.
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
'422':
description: The URL hostname could not be resolved.
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
'429':
description: Rate limit exceeded.
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
'503':
description: Service temporarily busy. Retry shortly.
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
'504':
description: Request timed out.
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
components:
parameters:
url:
name: url
in: query
required: true
description: >-
The website URL to process. `https://` is prepended automatically if no
protocol is provided.
schema:
type: string
example: https://stripe.com
mapLimit:
name: limit
in: query
required: false
description: Maximum number of links to return.
schema:
type: integer
minimum: 1
maximum: 5000
default: 1000
mapSearch:
name: search
in: query
required: false
description: >-
Rank and filter results by relevance to this query. Links that match
nothing are dropped.
schema:
type: string
example: pricing
mapSitemap:
name: sitemap
in: query
required: false
description: >-
Which sources to use. `include` merges sitemap entries with on-page
links, `skip` uses on-page links only, `only` uses sitemaps only.
schema:
type: string
enum:
- include
- skip
- only
default: include
mapIncludeSubdomains:
name: include_subdomains
in: query
required: false
description: Include links on subdomains of the target.
schema:
type: boolean
default: true
mapIgnoreQueryParams:
name: ignore_query_params
in: query
required: false
description: Treat URLs that differ only by query string as one page.
schema:
type: boolean
default: true
schemas:
MapData:
type: object
properties:
url:
type: string
domain:
type: string
nullable: true
links:
type: array
items:
$ref: '#/components/schemas/MapLinkItem'
count:
type: integer
total_discovered:
type: integer
description: Links found before `limit` was applied.
sources:
type: object
properties:
sitemap:
type: integer
links:
type: integer
sitemaps_used:
type: array
items:
type: string
Meta:
type: object
properties:
requestId:
type: string
format: uuid
description: Unique identifier for this request.
example: a3f2c1d4-7b6e-4f2a-9c1d-8e3f2a1b4c5d
durationMs:
type: integer
nullable: true
description: Time taken to process the request in milliseconds.
example: 1842
required:
- requestId
- durationMs
ErrorResponse:
type: object
properties:
success:
type: boolean
enum:
- false
error:
type: string
description: Human-readable error message.
example: Credit limit exceeded
meta:
$ref: '#/components/schemas/Meta'
required:
- success
- error
- meta
MapLinkItem:
type: object
properties:
url:
type: string
title:
type: string
nullable: true
description:
type: string
nullable: true
lastmod:
type: string
nullable: true
description: From the sitemap, when the site publishes one.
source:
type: string
enum:
- sitemap
- links
description: Where the URL was discovered.
securitySchemes:
apiKey:
type: apiKey
in: header
name: X-API-Key
description: Your Prefetch API key. Obtain one from the dashboard.
````