Inventory
URL inventory and redirect map
Every legacy URL is audited. 301 redirects preserve link equity and prevent legacy backlinks from becoming 404s.
Module 03 · Website Agentic Optimization
The foundation. Makes your site readable + trusted by AI crawlers.
AI engines do not click into your site — they read the structured data on first crawl.
Schema.org markup (Organization, Person, Article, FAQ, Speakable, mentions). llms.txt + robots.ai. Core Web Vitals. AI-crawler accessibility. Optional full migration from WordPress / Webflow / Tilda. The technical layer that lets Hermes + ContentOS actually compound.
Website Agentic Optimization is the foundation agent in the agentic workspace: it makes every page easy for the engines to quote.
Fetch
Structure
Quote
Crawler proof
- Schema
- llms.txt
- robots.ai
- CWV
Crawled by
- ChatGPT
- Claude
- Perplexity
- Gemini
- Grok
- DeepSeek
- Kimi
- Google AIO
- Copilot
Open-source Lite version
Audit whether AI can reach, choose, and quote your page. Upgrade when you need fixes shipped with proof.
AEO Site Audit Lite is a free MIT-licensed local audit for the three gates an AI engine checks before citing a page: Fetchable → Chosen → Extractable.
It runs offline with zero dependencies and no required API keys, then returns a prioritized fix list. Use Lite to inspect the surface yourself. Use the Website agent when the fixes must be implemented, rescanned, and tied to proof.
Lite
Free local audit
Fetchable → Chosen → Extractable checks, prioritized fixes, MIT license, zero deps, no API keys.
Website agent
Implemented fixes with proof
Schema, crawler access, internal links, llms.txt, performance fixes, rescans, and proof packets for the workspace loop.
Why the foundation matters
You can ship perfect content into a broken foundation. It will not get cited.
AI engines read the source, not the landing-page polish.
They send a crawler, read structured data and page text in one pass, then decide whether the page is citation-worthy.
A strong page can still remain invisible.
If robots.txt blocks crawlers, schema is missing, llms.txt does not exist, or Core Web Vitals fail, the work ContentOS ships has nowhere to compound.
Where teams go wrong
Technical SEO stops too early when AI crawlers become the reader.
Classic SEO checks stop before the AI-source test.
The page needs a path the crawler can fetch, the model can parse, and the answer can cite.
- Make the page fetchable for Google, OpenAI, Anthropic, Perplexity, and Microsoft crawlers.
- Make the entity graph clear with Organization, Person, Article, FAQ, and Breadcrumb schema.
- Make the answer extractable with clean headings, concise claims, and source-ready snippets.
- Rescan with Hermes so the technical fix is tied to citation lift, not opinion.
Schema.org
Full markup, not just Organization
Organization (with sameAs entity graph), Person (with knowsAbout + sameAs LinkedIn/GitHub), Article (with author + datePublished + speakable + wordCount), FAQPage (with Question/Answer), BreadcrumbList, SpeakableSpecification, mentions array per page for entity disambiguation.
AI accessibility
llms.txt + robots.ai + AI-crawler allowlist
llms.txt as the AI-readable site summary. llms-full.txt with the long-form corpus. robots.ai allowing ClaudeBot, Gptbot, PerplexityBot, OAI-SearchBot, Google-Extended explicitly. Server logs surface crawler hit count as a leading indicator of citation lift.
Performance
Core Web Vitals + image CDN
INP / LCP / CLS targets baked into the build pipeline. Image CDN migration (Sanity, Cloudflare R2). Font preload. CSS bundle discipline. No client-side hydration on static pages. Lighthouse 95+ on every shipped page.
Migration scope · what a Done-for-you engagement covers
WordPress → Astro / Next.js / Hugo. Every URL traceable, every signal preserved.
Content
Content migration
WP REST API or MDX exports move into content collections with EN, RU, and AR routes, frontmatter, body, media, and canonical mapping preserved.
Schema
Schema upgrade
Every page receives the full schema.org graph: Article, FAQ, Person, Service, BreadcrumbList, and the entity signals needed for AI citation.
AI files
llms.txt and sitemap rebuild
Sitemap index, locale sitemaps, lastmod data, llms.txt, and llms-full.txt are regenerated from the current corpus.
Crawlers
AI-crawler verification
robots.ai and allowlists are deployed, then server logs are checked for first-crawl events from major AI bots.
Performance
Performance budget
Representative pages are checked against Lighthouse 95+ and Core Web Vitals budgets before launch.
Lift
Hermes baseline and re-measure
Citation share is measured before migration and again at day 30 and day 60, with a target lift tied to the technical upgrade.
Reference dogfood
We ran this playbook on humanswith.ai itself.
787 URLs, 143 blog posts, 35 case studies, EN+RU+AR locales. Two-month rebuild, every redirect preserved, every schema upgraded.
Read the case →FAQ
Website Agentic Optimization — common questions.
What is the difference between traditional SEO and Website Agentic Optimization?
Traditional SEO optimises for Google ranking: keywords, internal links, page speed, backlinks. Website Agentic Optimization adds the layer AI engines need to TRUST and CITE you: schema.org markup (Organization, Person, Article, FAQ, Speakable, mentions), llms.txt declaration, robots.ai allowlist, AI-crawler accessibility, canonical entity graph. AI engines do not click into your site — they read the structured data on first crawl and re-cite based on it.
Is llms.txt actually used by AI engines?
OpenAI, Anthropic, and Perplexity have all signalled support; adoption is partial but growing fast in 2026. Even when the engine ignores llms.txt directly, the file forces you to maintain a structured public summary of your site — which the AI crawler reads anyway. Cost to ship: ~30 min. Cost to skip: invisible re-crawl signal.
Which schema.org types matter most for AEO?
In order of citation impact: Organization (with sameAs entity graph), Person (with knowsAbout + sameAs), Article (with author + datePublished + speakable), FAQPage (with Question/Answer), BreadcrumbList (with itemListElement), SpeakableSpecification (cssSelector hint for voice). Most sites have Organization + Article but skip Speakable + mentions — the two with highest AEO leverage today.
Do you handle the migration from WordPress / Tilda / Webflow?
Yes — full WP → Astro/Next.js/Hugo migration on Done-for-you tier. We migrated humanswith.ai itself in 2026 (787 legacy URLs, 143 blog posts, 35 case studies, EN+RU+AR locales) — every redirect preserved, every schema upgraded, every URL traceable. The migration runbook is our own dogfood case.
What about Core Web Vitals?
Part of the package on Scale + Done-for-you. INP / LCP / CLS targets baked into the build pipeline. Image CDN migration, font preload, CSS bundle discipline, no client-side hydration on static pages. Lighthouse 95+ on every shipped page (vs. WP averages of 50-70).
Can you fix my schema without a full migration?
Yes — schema-only engagement on Done-for-you ($5K+). Schema audit + JSON-LD generation + integration into your existing CMS (WP plugin, Webflow embed, Shopify metafield, custom). No site migration required. Output: validated structured data on every page within 4-6 weeks.
How do you measure technical-SEO impact?
Three signals: (1) Hermes citation lift after schema deploy (target +15% in 60 days); (2) Google Search Console Rich Results coverage for FAQ/Article/Breadcrumb (target 100% valid); (3) AI-crawler hit count from server logs (Gptbot, ClaudeBot, PerplexityBot, OAI-SearchBot — target 50+ hits/week post-deploy). All three reported monthly.
Audit your foundation. Free.
30 minutes. We run Lighthouse + schema validation + AI-crawler accessibility on your site.
You arrive to a per-engine technical-readiness map, the closeable gaps, and an honest read on whether full migration is worth the lift vs. schema-only fix.