---
description: New dashboard features to help you understand and optimize how AI agents interact with your content.
title: Tools to prepare your site for the agentic Internet
image: https://developers.cloudflare.com/changelog/post/2026-04-17-tools-for-agentic-internet/og.png?v=450058e05e35a671
---

[Skip to content](#main-content)

[View RSS feeds](https://developers.cloudflare.com/fundamentals/new-features/available-rss-feeds/) [ Subscribe to RSS](https://developers.cloudflare.com/changelog/rss/index.xml)

[Back to all posts](https://developers.cloudflare.com/changelog)

April 17, 2026

## Tools to prepare your site for the agentic Internet

[AI Crawl Control](https://developers.cloudflare.com/ai-crawl-control/)

Copy as Markdown| [View as Markdown](https://developers.cloudflare.com/changelog/post/2026-04-17-tools-for-agentic-internet/index.md)| [Agent setup](https://developers.cloudflare.com/agent-setup/)

AI Crawl Control now includes new tools to help you prepare your site for the agentic Internet—a web where AI agents are first-class citizens that discover and interact with content differently than human visitors.

#### Content Format insights

The **Metrics** tab now includes a **Content Format** chart showing what content types AI systems request versus what your origin serves. Understanding these patterns helps you optimize content delivery for both human and agent consumption.

#### Directives tab (formerly Robots.txt)

The **Robots.txt** tab has been renamed to **Directives** and now includes a link to check your site's [Agent Readiness ↗︎](https://isitagentready.com) score.

Refer to our [blog post on preparing for the agentic Internet ↗︎](https://blog.cloudflare.com/agent-readiness/) for more on why these capabilities matter.

```json
{"@context":"https://schema.org","@type":"BlogPosting","@id":"https://developers.cloudflare.com/changelog/post/2026-04-17-tools-for-agentic-internet/#page","headline":"Tools to prepare your site for the agentic Internet","description":"New dashboard features to help you understand and optimize how AI agents interact with your content.","url":"https://developers.cloudflare.com/changelog/post/2026-04-17-tools-for-agentic-internet/","inLanguage":"en","image":"https://developers.cloudflare.com/changelog/post/2026-04-17-tools-for-agentic-internet/og.png?v=450058e05e35a671","dateModified":"2026-04-17","datePublished":"2026-04-17","publisher":{"@type":"Organization","name":"Cloudflare","description":"One platform for your apps, agents, and workforce. Build, secure, and scale without managing infrastructure","url":"https://www.cloudflare.com/","sameAs":["https://github.com/cloudflare","https://www.linkedin.com/company/cloudflare","https://x.com/cloudflare"],"logo":{"@type":"ImageObject","url":"https://developers.cloudflare.com/logo.svg"},"address":{"@type":"PostalAddress","streetAddress":"101 Townsend St","addressLocality":"San Francisco","addressRegion":"CA","postalCode":"94107","addressCountry":"US"},"contactPoint":[{"@type":"ContactPoint","contactType":"Customer Support","url":"https://support.cloudflare.com/","availableLanguage":["English"]},{"@type":"ContactPoint","contactType":"Sales","url":"https://www.cloudflare.com/contact/","availableLanguage":["English"]}]},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"}}
```