Skip to main content
Fern's agent scores vs Cloudflare's Is Your Site Agent-Ready?
Last week, Fern published Agent Score, a tool to check the agent-friendliness of documentation portals in collaboration with Dachary Carey. Cloudflare now has a similar offering called Is Your Site Agent-Ready?. In theory, both tools should give similar results since they are both checking for agent-friendliness. However, the results are quite different. Why did that happen? That’s what I set out to find out.

​
Background

Fern publishes a leaderboard of the top documentation portals based on their agent-friendliness. Of these, I picked the top 10 portals and ran them through Cloudflare’s Is Your Site Agent-Ready? tool. Here are the top 10 portals from Fern’s leaderboard and their scores:
#Docs PortalFern’s Agent Score
1Agora100
2Promptless100
3Requesty99
4Ollang99
5Bolt99
6Resend99
7Granola99
8Feather98
9Runpod98
10Skyflow97

​
Approach

Cloudflare’s Is Your Site Agent-Ready? tool offers three types of scans:
  1. All checks: Run all default checks. Best for comprehensive audits.
  2. Content Site: Run checks relevant to content sites. Focuses on discoverability and content accessibility.
  3. API / Application: Run checks relevant to API / Application sites, except commerce.
To ensure fairness, I ran the content site scan for all 10 portals. As of this writing, that involves the following checks:
Cloudflare content site checks

​
Results

The results were quite surprising. None of the portals scored above 90, with Resend coming closest at 83. Here is how the scores compare side by side:
#Docs PortalFern’s Agent ScoreCloudflare’s Is Your Site Agent-Ready
1Agora10017
2Promptless10067
3Requesty9967
4Ollang9967
5Bolt9967
6Resend9983
7Granola9967
8Feather9867
9Runpod9867
10Skyflow9767
Cloudflare’s own developer documentation scores 100 on Is Your Site Agent-Ready? and 97 on Fern’s Agent Score.

​
Understanding the results

The scores are clearly different, but why are they so different? To find out, I looked at how these tools suggest improvements. Both Fern and Cloudflare offer an agent prompt to fix the issues they flag. The idea was to understand common patterns that both tools flag and where their suggestions diverge. I copied the suggestions from Fern and Cloudflare, exported them as Markdown, and ran them through Claude Sonnet 4.6 Thinking via Perplexity.
I don’t have an active subscription to Claude Opus 4.7 or GPT-5 Pro. If you do and are interested in running the results through other models, see Resources.

Prompt

Fern_Cloudflare_agent_score_comparison.csv contains comparison between the agent scores from Fern’s agent score checker (https://buildwithfern.com/agent-score) and Cloudflare’s Is Your Site Agent-Ready? (https://isitagentready.com/).Cloudflare_agent_readiness_improvement_suggestions.md and Fern_agent_readiness_improvement_suggestions.md contain suggestions from Cloudflare and Fern to improve agent scores respectively.Based on this data, find out:
  1. Why do the scores differ vastly?
  2. What is the common pattern in the suggestions from Fern and Cloudflare?
  3. Which change has the greatest ROI?
  4. What are the key takeaways? Analyze the findings and share an executive summary.

Executive summary

The two tools measure fundamentally different dimensions of “agent readiness”, which is why scores diverge so dramatically — Fern scores cluster in the 97–100 range while Cloudflare scores range from 17 to 83. The gap is not a measurement error; it reflects two distinct philosophies: content accessibility vs. web infrastructure governance.
The tools operate on completely different evaluation frameworks:Fern (buildwithfern.com/agent-score) measures how well an AI agent can read and consume your documentation content. Its checks focus on:
  • Whether llms.txt exists and is properly structured
  • Whether pages serve clean Markdown (.md URLs, content negotiation via Accept: text/markdown)
  • Whether page content starts early (low boilerplate ratio)
  • Whether Markdown and HTML representations are in parity (no hidden JS-only content)
  • Whether sitemaps exist for discovery
Cloudflare (isitagentready.com) measures web infrastructure and governance for agent interaction. Its checks focus on:
  • Whether robots.txt exists with explicit AI crawler rules (GPTBot, Claude-Web, Google-Extended)
  • Whether the site supports Markdown for Agents (HTTP content negotiation returning text/markdown)
  • Whether RFC 8288 Link headers are present for agent discovery
  • Whether Content Signals (ai-train=no/yes, ai-input=no/yes) are declared in robots.txt
  • Whether sitemap.xml exists with canonical URLs
In short: Fern asks Can an agent read your docs?, while Cloudflare asks Does your site talk to agents like a good web citizen? Sites built on modern docs platforms naturally score high on Fern’s content checks, but almost universally fail Cloudflare’s infrastructure-level governance checks — Content Signals are missing on all 10 sites, and Link headers are missing or inadequate on 9/10 sites, with Resend being the only genuine pass on the latter.
Both tools converge on two themes: structured content discoverability and clean machine-readable responses.
Suggestion ThemeFern’s VersionCloudflare’s Version
Machine-readable contentServe .md URLs; return Markdown on Accept: text/markdownEnable “Markdown for Agents” via HTTP content negotiation
Sitemap / crawlabilityGenerate /sitemap.xml listing all doc pagesGenerate /sitemap.xml and reference it from robots.txt
llms.txt / agent indexCreate proper /llms.txt with blockquote summariesN/A (Cloudflare focuses on robots.txt AI rules instead)
Content parityEnsure HTML and Markdown have the same content (no JS-only tabs)Ensure no auth walls block agent crawlers
Signal declarationllms.txt directive on all pagesContent-Signal directives for ai-train, ai-input in robots.txt
The single most universal gap across both tools is the absence of machine-readable content negotiation — every site either lacks .md URLs, a proper Markdown response header, or both.
Enabling Markdown content negotiation and addingContent Signals to robots.txt offer the highest ROI, and here’s why:
  • Markdown content negotiation (Accept: text/markdown): Both Fern and Cloudflare flag this. On the Fern side, it eliminates the Markdown Content Parity warning (affects 2–3 pages on most sites). On Cloudflare’s side, it converts a hard fail into a pass for the “Markdown for Agents” check. Enabling this via a CDN/middleware rule is typically a single configuration change.
  • Content Signals in robots.txt: This is flagged as a failing check on every single site across all 10 in Cloudflare’s report. It requires a one-line edit to an existing file (Content-Signal: ai-train=no, search=yes, ai-input=no) — zero engineering effort, but unlocks Cloudflare’s governance score immediately.
  • RFC 8288 Link headers: Flagged on 8/10 sites by Cloudflare. Adding Link: </.well-known/api-catalog>; rel="api-catalog" at the server/CDN level is a one-time header rule that immediately improves agent discovery.
The trifecta of Markdown negotiation + Content Signals + Link headers can be implemented with no content changes, purely at the infrastructure layer, and would boost Cloudflare scores from 67 → ~90+ for most sites.
  • Two tools, two different definitions of “ready”: Fern assesses content quality and LLM-specific formats; Cloudflare assesses governance, HTTP standards, and web infrastructure. You need both to be truly agent-ready — a site can score 99 on Fern while being woefully unprepared on the Cloudflare dimensions.
  • Agora is the most egregious outlier: It scores 100 on Fern but only 17 on Cloudflare — it has excellent documentation content but zero infrastructure governance (no robots.txt, no sitemap, no AI rules). A perfect content score is meaningless if agents can’t determine whether they’re allowed to use the content.
  • The “67 plateau” on Cloudflare is structural: Eight out of ten sites share the exact same Cloudflare score of 67. This strongly suggests they’re all passing the same baseline checks (e.g., robots.txt exists, sitemap exists) but uniformly failing the newer governance checks (Content Signals, Link headers). This is an industry-wide blind spot, not a site-specific issue.
  • llms.txt quality is a near-universal Fern warning: Even sites scoring 98–99 on Fern have llms.txt files that are missing the blockquote summary required by the spec. Fixing this is trivial — add a > Brief summary of the docs line to the file.
  • The highest-scoring site overall is Resend (99 Fern, 83 Cloudflare), suggesting it has implemented more of the infrastructure-level signals. It serves as the benchmark to emulate.

​
Conclusion

As AI agents increasingly become the primary consumers of documentation, being truly agent-ready must evolve beyond just writing clean, well-structured content. While having high-quality documentation is foundational (as measured by Fern), adopting proper web infrastructure protocols and governance (as measured by Cloudflare) is the next major frontier for developer platforms. The current industry-wide blind spot regarding machine-readable content negotiation and Content Signals presents a massive opportunity. Documentation teams that proactively bridge this gap by aligning both their content structures and their infrastructure layer won’t just score higher on these evaluations, they’ll ensure their APIs are the easiest for autonomous agents to discover, understand, and integrate with.
Dachary Carey has now published a companion investigation where she measures the adoption of Cloudflare’s standards against actual agent traffic to highlight how agents consume docs today.
The platforms that will ultimately win in the AI era are the ones that learn to communicate with agents just as fluently as they do with human developers.

​
Resources

Test with Fern

Run your docs through Fern’s Agent Score and share your results.

Test with Cloudflare

Check your site’s infrastructure readiness with Cloudflare’s Is Your Site Agent-Ready?

Analysis resources

Google Drive folder with all the data and files used in this analysis.

​
Share your feedback

Did you find this useful? Have a different take on the results? I’d love to hear from you.

Contact me

Reach out with feedback, corrections, or suggestions.

Connect on LinkedIn

Follow me on LinkedIn for more posts like this.