CrawlCheck

Connect it to your AI

Claude Code

claude mcp add --transport http crawlcheck https://crawlcheck.io/mcp

Claude, ChatGPT, Cursor and other apps

Add it as a custom connector with this URL (Streamable HTTP):

https://crawlcheck.io/mcp

Cursor, VS Code and other apps (mcp.json)

{
  "mcpServers": {
    "crawlcheck": {
      "url": "https://crawlcheck.io/mcp"
    }
  }
}

Tools it offers

25 tools, as the server itself reported during our check on Oct 31, 2026.

• preflightCall BEFORE reading, citing, connecting to or transacting with a domain. Returns a signed decision: allow, warn, require_confirmation, block or unsupported, with every step that led there and the resolve answer it was ma
• best_sourceCall when choosing WHICH business or site to cite, recommend or call. Give domains (2-10 candidate sites you found) or trade and city for a measured market. Returns the candidates ranked by what CrawlCheck verified (pref
• resolve_domainCall before fetching from, citing, connecting to or transacting with a domain. Returns one signed answer: what the domain claims, what it actually serves each crawler identity, who controls it, what CrawlCheck's publishe
• field_checkCall before using a phone number, address, opening hours or service area you hold for a business. One signed verdict per business field. field=phone|address|hours|service_area and value= the value you hold. phone: matche
• scan_domainScan a public domain as several crawler identities and return the graded record: grade, AI-visibility score, per-section scores, findings with evidence, and whether the origin refused CrawlCheck. One scan takes 5-15 seco
• list_verified_capabilitiesWhat a domain declares agents can do (OpenAPI operations, MCP tools, A2A skills, API catalog entries), each classed by safety (read_only_public up to financial), and which ones CrawlCheck actually called and confirmed -
• mcp_serversWhich remote MCP servers the official MCP Registry lists on a domain, and what each endpoint actually answered when CrawlCheck sent the MCP handshake (initialized, auth_required, unreachable, tool count, tool-list drift)
• verify_crawler_logGiven raw web-server access-log lines, decide for each line that names a crawler (GPTBot, ClaudeBot, Googlebot, PerplexityBot...) whether the source IP falls inside that operator's published ranges. Returns verified, spo
• corpus_stateCoverage of CrawlCheck's public dataset: how many distinct domains have been measured and how they split by platform, rendering, size, language and kind, with the strata that are still under-sampled.
• resolve_robotsResolve every named answer engine, search index and training crawler against a domain's robots.txt the way a crawler does: most-specific group only, longest match, allow wins a tie. Shows when a Disallow under * does not
• draft_llmsDraft an llms.txt from the domain's own homepage and up to 25 declared pages, using their titles and descriptions. A draft for the owner to cut down, not a publication.
• fix_robotsThe domain's served robots.txt, corrected: * group rules copied into named groups that lacked them (shadowing), a Sitemap line added when missing, a minimal replacement when the served file was HTML. Every change is list
• fix_entitymapA starter entitymap.json built from the domain's latest scan record: name, phone, coordinates and declared service areas as entities with SERVES relations. Fields the page never stated are marked TODO, never guessed. Nee
• video_checkOne YouTube video against the video anchor model: metadata, chapters, captions and storyboard where observed, twelve checks each naming what it reads.
• video_channelEvery upload on a YouTube channel, read 50 at a time, sorted by views. The upload list is free; the per-check tally needs a Watch licence and is not offered here.
• video_siteThe site half of video: homepage plus up to 60 sitemap pages read for YouTube embeds, facades and page-builder widgets, VideoObject nodes and their required properties, and - with a handle - how many of the channel's vid
• registry_lookupWhether a domain is in CrawlCheck's registry and what it serves: llms.txt, agents.md, a media kit at /.well-known/media-kit.json, a reciprocity-tested entity graph, an AI access policy that names crawlers, and agent-call
• crawler_pathWhere each crawler's path into a domain dies, as an observed data flow: edge decision, robots.txt as a file, robots.txt rules, then the page, JSON-LD, sitemap, llms.txt and entitymap.json it reached. Give agent (e.g. Cla
• public_countsThe figures CrawlCheck quotes about itself, read from the source: sites_measured, domains_in_corpus, crawler_visits (a rolling window), sections, sections_scored, findings_published, guides_published, with an at timestam
• telemetryVerified-crawler traffic observed at crawlcheck.io itself: which named agents arrived, how many claims were confirmed against operator ranges, how many were forged.
• fix_mostly_codeA fix plan for PAGE_IS_MOSTLY_CODE: the homepage's measured bytes by category, the largest blocks with what to do with each, ordered steps with the text ratio after each, and the step at which the detector would stop fir
• fix_stale_cacheA fix plan for STALE_CACHE_SERVED: the homepage fetched now, its Age, the cache layers that name themselves in the headers in purge order (innermost first), the HTML lifetime each response declares, a fresh-read comparis
• fix_machine_chainA fix plan for MACHINE_CHAIN_HEAVY: the files a crawler reads before the page and the 404s absent agent files return, then measured reductions (llms.txt to an index, a sitemap index, robots.txt comments and duplicate gro
• explain_findingWhy CrawlCheck decided a finding: the rule and its revision, the exact fetches it compared (identity, status, bytes, sha256), when the condition held on the site, and a Merkle proof that the decision is inside the signed
• machine_recordThe unified machine record for one observation: subject, grade, findings (the top one in full without a licence), capabilities with verification, and the evidence roots, manifest and verifier links that let anyone check

Questions