Cloudflare and OpenAI pilot shares network signals to help ChatGPT search discover hundreds of millions of new pages
Cloudflare and OpenAI report early results from a research pilot launched in July 2026. The pilot tests whether signals from Cloudflare's network can help OpenAI's search crawler find more useful pages for ChatGPT Search. Using a cohort of about 1% of Cloudflare's network, OpenAI discovered and indexed hundreds of millions of previously unknown web pages. This is product and partnership news, not a security incident. It does include a defensive element: Cloudflare is testing signals that flag individual harmful pages, such as a fake login form planted on a legitimate site.
Technical Details
- Cloudflare supplies metadata about the web, not page content, for search and discovery. The article says this is not shared for model training. A signal about a page does not grant permission to access it.
- Filtering signals identify non-existent pages that bots repeatedly request, sites that redirect most addresses to their homepage, and duplicate content. This cuts wasted crawler fetches.
- Change signals tell crawlers when a page is worth revisiting.
- Early results suggest page-level harm signals can help keep malicious pages out of an index without penalizing the whole domain.
- Crawling may initially increase as newly found pages are evaluated and indexed.
Remediation
No remediation applies. Site owners can use Cloudflare's AI Crawl Control to understand and manage how AI systems access their sites.
Linha do tempo
-
Cloudflare and OpenAI pilot shares network signals to help ChatGPT search discover hundreds of millions of new pages
ler a análise desta fonte ocultar
Cloudflare and OpenAI report early results from a research pilot launched in July 2026. The pilot tests whether signals from Cloudflare's network can help OpenAI's search crawler find more useful pages for ChatGPT Search. Using a cohort of about 1% of Cloudflare's network, OpenAI discovered and indexed hundreds of millions of previously unknown web pages. This is product and partnership news, not a security incident. It does include a defensive element: Cloudflare is testing signals that flag individual harmful pages, such as a fake login form planted on a legitimate site.
Technical Details
- Cloudflare supplies metadata about the web, not page content, for search and discovery. The article says this is not shared for model training. A signal about a page does not grant permission to access it.
- Filtering signals identify non-existent pages that bots repeatedly request, sites that redirect most addresses to their homepage, and duplicate content. This cuts wasted crawler fetches.
- Change signals tell crawlers when a page is worth revisiting.
- Early results suggest page-level harm signals can help keep malicious pages out of an index without penalizing the whole domain.
- Crawling may initially increase as newly found pages are evaluated and indexed.
Remediation
No remediation applies. Site owners can use Cloudflare's AI Crawl Control to understand and manage how AI systems access their sites.