Skip to main content

Endpoint

Authentication

Required.

Request body

agentId
string
required
The agent ID.
url
string
required
The URL to crawl. Must be a valid HTTP or HTTPS URL.
mode
string
required
Crawl mode: crawl (recursive), sitemap (discover from sitemap), or single (one page only).
maxPages
number
Maximum number of pages to crawl. Range: 1–100. Default: 50.
maxDepth
number
Maximum crawl depth for link following. Range: 1–3. Default: 2.

Response

success
boolean
webSource
object
The created web source object (see List Web Sources for the full shape).

Limits

  • Maximum 10 web sources per agent.

Example