vaaya / discover / crawl site

Crawl site for AI agents

Crawl a whole site from a seed URL, following links — index docs, catalogs, or a competitor across many pages.

from 1¢

What it's for

Crawl a whole site starting from one seed URL, following its links.
Good for indexing docs, a product catalog, or a competitor site.
Cap how many pages it follows so it doesn't run away.
Comes back as clean markdown per page, ready to read or search.

How to ask your agent

Connect Vaaya, then just say it in plain language. For example:

Crawl docs.stripe.com/payments, up to 20 pages.

Crawl our competitor's whole docs site and summarize what changed.

What you get back

{
  "status": "completed",
  "total": 14,
  "data": [
    { "markdown": "# Payments\n\n…", "metadata": { "sourceURL": "https://docs.stripe.com/payments", "title": "Payments" } },
    { "markdown": "# Payment methods\n\n…", "metadata": { "sourceURL": "https://docs.stripe.com/payments/payment-methods", "title": "Payment methods" } }
  ]
}

Pricing

from ~1¢ per page crawled. Billed per call in cents, capped by your max, every call on the ledger. Failed calls are not billed.

Give your agents this.

Connect Vaaya once, then describe the outcome. It picks the right capability, runs it, and pays per call.

npx @vaaya/mcp install