-
Three weeks on, 80 of 82 x402 sites still block AI training crawlers
On 18 October 2026 we repeated our crawler check on the x402 Bazaar. Of 82 sites that blocked AI training crawlers and are still up, 80 still do.
Read more -
A month later, 98 of 104 AI-built sites look the same to a crawler
On 18 October 2026 we checked the same 120 AI-built sites again. 98 of 104 gave the same answers; 3 vanished as their domains came up for renewal.
Read more -
Many AI-built sites have a finished link preview and an empty page
On 27 September 2026, 48 of 105 AI-built sites showed a complete link preview card, while a crawler found almost no text on the page behind it.
Read more -
Half of x402 services have no front page an AI crawler can read
On 25 September 2026 we opened the front page of all 2,060 hosts in the x402 Bazaar. 46% answered with a page a crawler can read.
Read more -
Five reasons AI crawlers cannot read your site, and how to fix each
Five problems we keep finding on AI-built and x402 sites: empty HTML, soft 404s, two hostnames, no sitemap, a firewall. A curl test and a fix for each.
Read more -
One in eleven x402 sites blocks AI training crawlers
On 27 September 2026, 86 of 908 x402 Bazaar sites refused the AI training crawlers with a 403, while 83 of them still let answer crawlers in.
Read more -
We fetched 104 AI-built sites the way GPTBot does. 61% were empty.
61% of 104 live sites made with AI app builders were empty to a crawler that runs no JavaScript. Our data, a control group and five fixes.
Read more
New posts by feed: RSS (Atom)