402registry-check: the crawler behind 402registry

Why it visited your site

• Someone paid for a check of your site — usually its owner, or an AI agent working for them. The report goes only to whoever paid.
• Someone submitted your site for a free check through our public queue.
• We picked your site for the public catalog from public lists of sites built with AI tools.
• A site listed in the catalog is re-checked for free about every 30 days, so the date on its entry stays true.
• A customer asked for scheduled re-checks of their own site.

What one check requests

• The page itself, following at most 5 redirects; then the other form of the domain (with and without www) and the http:// version, one hop each.
• /robots.txt, /sitemap.xml, /llms.txt, and up to 2 URLs listed in your sitemap.
• One URL that should not exist — /402registry-probe- followed by random characters — to see whether a missing page returns a real 404.
• The same page three more times with the bare user-agent names GPTBot, ClaudeBot, PerplexityBot, to see whether your firewall blocks those AI crawlers. Those three requests come from us, not from the companies behind those crawlers.

At most 18 requests per check, each with a 6-second timeout, reading at most 1.5 MB. No JavaScript is run, no form is submitted, nothing is posted to your site.

Where it comes from

We run on Cloudflare Workers, so requests come from Cloudflare’s network. There is no fixed IP range to allow-list — identify us by the user-agent.

Opting out

• Add this to your robots.txt:

  User-agent: 402registry-check
  Disallow: /

  We read robots.txt on every check. While this rule is in place your site is not added to the free catalog, and an existing entry is removed at its next check. A report someone has already paid for is still produced.
• Or write to abuse@402registry.com and we add your site to our opt-out list: it is never listed.

Other names you may see

• 402registry-notify — change notifications we POST only to webhook addresses our customers registered themselves.