Skip to content
Answer Stack
Open menu

What CMS features matter for AEO?

✓ Verified Last reviewed by AnswerStack Next review due Nov 9, 2026

Every claim is sourced below

When you choose or audit a CMS for AEO, we'd check four practical controls: how the page is delivered, which crawlers can reach it, what the templates output, and how easily your team can change a page. They affect whether a page can be fetched and read, and how quickly a wrong fact gets corrected. Google's AI features in Search rely on the same crawling and indexing as Search, so its technical SEO advice still applies to them.

Control What it affects What to check
Page delivery Whether the text is in the HTML a crawler receives View the page source and search for a sentence from the body copy
Crawl and index rules Which crawlers are asked to stay out, and which pages ask not to be listed Where robots.txt is edited, whether rules can name a crawler, and noindex per page
Templates The structure and markup that pages share Markup generated from the content, with no duplicates
Editing How a wrong fact gets fixed Whether the person who knows the subject can update a page without a developer

What a crawler gets when it asks for the page

The first thing a CMS decides is what comes back when something requests a URL. A CMS that builds the page on the server returns HTML with the words already in it. A site that builds some or all of the page in the browser may leave part of the text out of that HTML, and the missing part appears after a script runs.

Google can handle both. Its JavaScript SEO documentation says Googlebot queues pages for rendering and runs their JavaScript in a headless Chromium browser. The same page still recommends server-side rendering or pre-rendering, partly because, in Google's words, "not all bots can run JavaScript." Text that's already in the server's HTML doesn't depend on a crawler running JavaScript. So delivery belongs on the list when you choose or audit a CMS.

You can check this on your own site. Open a page, view its source, and search for a sentence from the body copy. If the sentence is there, that text is in the first response. If it isn't, treat that as a reason to inspect how the page renders. Google's URL Inspection tool shows the rendered HTML that Google works from.

A missing page should return a real 404 status code, because Googlebot reads status codes to learn that something went wrong. Request a URL that doesn't exist and see what status your CMS sends back. A "not found" message delivered with a 200 success status is the thing to fix.

Crawl and index controls

We'd want a CMS to let you edit robots.txt yourself, write separate rules for separate crawlers, and add a noindex tag to a single page.

Per-crawler rules matter because AI companies run more than one crawler. OpenAI's crawler documentation lists OAI-SearchBot, which surfaces sites in ChatGPT's search features, and GPTBot, which collects content that may be used to train its models. OpenAI says each setting is independent of the other, so a site can allow the search crawler and disallow the training crawler.

That makes two things worth checking. The first is whether your CMS lets you write a rule for a named crawler. The second is whether those rules are set somewhere else, such as a hosting layer or a proxy in front of the site. A rule that blocks every crawler in order to stop training also blocks OAI-SearchBot. OpenAI says sites opted out of that crawler won't be shown in ChatGPT search answers, though they can still appear as navigational links. Blocking a crawler you wanted is one of the biggest AEO mistakes to avoid.

HubSpot shows what these controls look like in a real product. Its documentation on keeping content out of search results describes a robots.txt editor in the SEO & Crawlers settings. You can set rules for all connected domains or override them for one domain. Each rule names a user agent, so per-crawler rules are possible. For a single page, you add a noindex meta tag to the head HTML field in that page's settings.

The two tools do different jobs, and HubSpot warns against combining them. A robots.txt rule tells a crawler not to fetch the page. A noindex tag sits on the page and asks a search engine not to list it. If robots.txt blocks the crawler, the crawler never sees the tag. So decide which one you mean before you use either. Block crawling for the crawlers you don't want, and use noindex for the pages you don't want listed.

What the templates output

Templates supply the structure that pages share: the heading levels, the canonical tag, the slots for a title and description, and any structured data. A mistake in a template repeats on each page that uses it.

Google's guide to generative AI features in Search recommends semantic HTML where possible, and adds that perfect code isn't required. In CMS terms, we'd want the editor to produce real headings and let you set the title and description for each page.

With structured data, look at what the CMS already outputs before you add anything. Google's guide says structured data isn't required for its AI features and that there's no special markup to add. It still recommends markup because it helps a page qualify for rich results. HubSpot covers blog posts natively. Its blog settings include a Structured Data switch that turns on BlogPosting markup for a whole blog, and the switch is on by default for blogs created after August 4, 2022. HubSpot also warns against turning it on when custom schema already exists, since duplicate markup can cause problems in search engines.

Markup that a template generates from the post's own fields changes when the post changes. Markup pasted into a page by hand stays as it was until somebody goes back to it.

Lean Labs' Schema Rocket is an example of the first kind, built on top of HubSpot. Schema Rocket adds markup beyond the native blog setting. Lean Labs describes it as mapping a company's entities and the relationships between them, generating JSON-LD, and giving a person or company the same identifier on every page where they appear. It regenerates the markup when a workflow triggers it after a content change. Whether you buy a tool or build the templates yourself, ask the same thing of the CMS: can markup be generated from the content and updated without a developer editing each page? We go further into markup in what role structured data plays in AEO.

How easily the content can be kept current

Answers go stale: a price changes, a product gets renamed, a process gains a step. When the person who knows the facts can edit the page in the CMS, a correction doesn't have to wait for a developer's time.

Lean Labs builds its client answer hubs in HubSpot for that reason. Its AEO service page says the content lives in HubSpot so the client's team can update answers without a developer. We describe that upkeep in how AEO works with existing HubSpot content.

Small template settings affect accuracy too. HubSpot's blog settings let you choose the format of the publish date, and the documentation notes a trap. If the format has the year typed in by hand, posts keep showing that year. The displayed date can then be wrong until someone replaces the typed year with the year token.

Check the plan as well as the platform

A feature the platform has isn't always a feature your subscription has. HubSpot's blog settings are available on all plans, but the same page lists exceptions. Changing a blog's root URL requires Marketing Hub Enterprise. Limiting a blog to access groups requires Content Hub Professional or Enterprise. Check any other platform the same way, and compare the plan you'd pay for. We cover HubSpot's product in what HubSpot CMS is and how Content Hub works and its top tier in what HubSpot CMS Enterprise includes.

What a CMS can't do

A CMS can make a page reachable and readable. It can't make the page worth citing. Google's guide says that a page meeting every requirement still isn't guaranteed to be crawled, indexed or served. It also says useful, unique content will likely matter more than any other suggestion in the guide, and that Google Search ignores llms.txt files. So where Google Search is concerned, llms.txt support isn't a reason to choose one CMS over another.

For the same reason, we'd inspect the gaps before changing platforms. Go through the four controls on your current site and name what's missing. Then see whether a setting, a template change or a plan upgrade closes each gap before you price a migration.

Sources

The rise of the AI crawler

Vercel (with MERJ)

Independent Verified Aug 9, 2026 Supports: No major AI crawler renders JavaScript; Gemini runs on Googlebot infrastructure; JS fetch shares of 11.50% and 23.84%; 34.82% of ChatGPT fetches on 404s and 14.36% on redirects vs 1.49% for Googlebot.

“The results consistently show that none of the major AI crawlers currently render JavaScript.”

Understand the JavaScript SEO basics

Google Search Central

Primary source Verified Aug 9, 2026 Supports: Crawl, render, index sequence; rendering queue delay; Google's recommendation of server-side or pre-rendering because not all bots run JavaScript.

“Keep in mind that server-side or pre-rendering is still a great idea because it makes your website faster for users and crawlers, and not all bots can run JavaScript.”

Streaming (Bots and crawlers)

Next.js documentation

Primary source Verified Aug 9, 2026 Supports: HTML-limited bots detected by user agent receive blocking metadata; htmlLimitedBots config; shells dependent on build-time data can fail for crawlers.

“HTML-limited bots and crawlers need metadata to be available in the <head> of the initial HTML. Next.js detects them by their user agent and waits for generateMetadata to resolve before streaming the page content.”

Introduction to Structured Data

Google

Primary source Verified Aug 9, 2026 Supports: JSON-LD is the recommended format; Google reads JSON-LD injected by JavaScript; structured data must describe information visible to the user.

“Google can read JSON-LD data when it is dynamically injected into the page's contents, such as by JavaScript code or embedded widgets.”

AI features and your website

Google Search Central

Primary source Verified Aug 9, 2026 Supports: Eligibility for AI Overviews and AI Mode is indexing plus snippet eligibility; no machine readable files, AI text files, or special schema.org markup required; Google-Extended controls training and grounding in Google's other systems.

“You don't need to create new machine readable files, AI text files, or markup to appear in these features.”

Optimizing your content for inclusion in AI search answers

Microsoft Advertising (Bing)

Primary source Verified Aug 9, 2026 Supports: AI assistants parse pages into pieces rather than reading them linearly; schema, clear headings, and modular layouts recommended; schema helps AI systems understand content.

“AI assistants don't read a page top to bottom like a person would.”

Introduction to robots.txt

Google Search Central

Primary source Verified Aug 9, 2026 Supports: Rules apply per user-agent group and per path; robots.txt is not an indexing control.

“it is not a mechanism for keeping a web page out of Google”

OpenAI crawlers (GPTBot, OAI-SearchBot, ChatGPT-User)

OpenAI

Primary source Verified Aug 9, 2026 Supports: GPTBot for training, OAI-SearchBot for ChatGPT search, ChatGPT-User for user-initiated visits; opting out of OAI-SearchBot removes a site from ChatGPT search answers.

“Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though can still appear as navigational links.”

Perplexity crawlers (PerplexityBot, Perplexity-User)

Perplexity

Primary source Verified Aug 9, 2026 Supports: PerplexityBot surfaces and links sites in search results; Perplexity-User is a user-initiated fetcher that generally ignores robots.txt.

“Since a user requested the fetch, this fetcher generally ignores robots.txt rules.”

The /llms.txt file

llmstxt.org (Jeremy Howard)

Primary source Verified Aug 9, 2026 Supports: Definition and purpose of llms.txt; v2 adoption notes; markdown alternates at the same URL with .md; rel=alternate type=text/markdown and rel=describedby, as link elements or an HTTP Link header.

“Agents are best served by concise, expert-level information gathered in a single, accessible location.”

Giving users choice with Cloudflare's new Content Signals Policy

Cloudflare

Primary source Verified Aug 9, 2026 Supports: The search, ai-input, and ai-train signals inside robots.txt; syntax; applied to more than 3.8 million domains; signals are preferences, not enforcement.

“It's important to remember that content signals express preferences; they are not technical countermeasures against scraping.”

Why we're rethinking cache for the AI era

Cloudflare

Independent Verified Aug 9, 2026 Supports: Over 10 billion AI-bot requests per week across Cloudflare's network; AI crawler unique-access ratio typically between 70% and 100%; effect on cache hit rates.

“AI crawler traffic has three main differentiating characteristics: high unique access ratio, content diversity, and crawling inefficiency.”

Large site owner's guide to managing your crawl budget

Google Search Central

Primary source Verified Aug 9, 2026 Supports: Crawl rate falls when latency rises or servers return 5xx or 429; soft 404 pages continue to be crawled and waste budget.

“If the site slows down (latency increases or response times become longer), or responds with server errors (5xx HTTP status codes) or rate-limiting signals (such as HTTP 429), the limit goes down and Google crawls less.”

Byline dates in Google Search results

Google Search Central

Primary source Verified Aug 9, 2026 Supports: datePublished and dateModified on CreativeWork subtypes; visible and structured dates must match; Google does not depend on a single date factor.

“Ensure that the date (and optional time and timezone) match between the equivalent user-visible and structured values.”

Template Hierarchy

WordPress Developer Resources

Primary source Verified Aug 9, 2026 Supports: WordPress selects a template per request from the template hierarchy to display the page.

“WordPress uses the query string to decide which template should be used to display the page.”

do_robots()

WordPress Developer Resources

Primary source Verified Aug 9, 2026 Supports: WordPress outputs a generated robots.txt disallowing /wp-admin/ while allowing admin-ajax.php, modified through the robots_txt filter.

“Displays the default robots.txt file content.”

Schema specification

Yoast

Primary source Verified Aug 9, 2026 Supports: Yoast SEO outputs a JSON-LD @graph describing WebPage, WebSite, and Organization on every page of a site running the plugin, with pieces cross-referenced by ID.

“The core of our approach is to output a "base script" - a @graph object rendered in JSON-LD - which describes the WebPage, the WebSite, and the Organization.”

Increased custom code character limit

Webflow

Primary source Verified Aug 9, 2026 Supports: Webflow allows up to 50,000 characters of custom code across site settings, page settings, code embed elements, and CMS rich text fields, on all paid sites and workspaces.

“You can now add up to 50,000 characters of custom code across Site settings, Page settings, Code Embed elements and CMS Rich text fields.”

Introduction to llms.txt and AEO

Webflow University

Primary source Verified Aug 9, 2026 Supports: Webflow's own guidance on hosting an llms.txt file at the site root as a manual plain text file, and its caveat that the file is a signal rather than a control.

“It's worth noting that llms.txt is not a guarantee that AI systems will follow its guidance”

Manage blog settings

HubSpot Knowledge Base

Primary source Verified Aug 9, 2026 Supports: Blog-level Structured Data toggle applying BlogPosting schema, on by default for blogs created after August 4, 2022, with a warning about duplicate schema.

“To turn on BlogPosting schema for the blog, toggle the Structured Data switch on.”

Prevent content from appearing in search results (customize your robots.txt file)

HubSpot Knowledge Base

Primary source Verified Aug 9, 2026 Supports: Location of HubSpot's robots.txt editor and its per-domain override behavior.

“Settings > Content > Pages > SEO & Crawlers”

Revision history

1 revision since publication
Originally published