How This Blog Is Built: Astro, Cloudflare Workers and SEO for the AI Era

The short answer

A static site on an edge network is the best value setup for a personal blog right now: Astro turns Markdown into very fast HTML, and Cloudflare Workers serves the static files from a global CDN with free TLS, plus a little server-side code where it’s needed. On top of that I added three things:

Analytics — humans and AI crawlers tracked separately
SEO — structured data, sitemap, canonical URLs, Open Graph
AIEO — making the content easy for AI engines to find, understand and cite

Architecture

Markdown (src/content/blog)
        │  astro build
        ▼
   dist/ static HTML ──► Cloudflare Workers Static Assets (global CDN)
                              │
                 Worker (run_worker_first)
                              │  writeDataPoint()
                              ▼
                 Workers Analytics Engine ──► SQL reports

Analytics in two layers

Layer	Tool	Sees	Misses
Client	Cloudflare Web Analytics	Human visitors, referrers, countries, Core Web Vitals	AI and search crawlers
Server	Worker + Analytics Engine	Every request (incl. AI crawlers), UTM tags, AI referrals	Time on page

The server-side classifier tags every request as human, ai-bot, search-bot or bot, and groups traffic sources into channels such as search, ai, social, referral and direct. ChatGPT, for example, appends ?utm_source=chatgpt.com to links, so those visits land in the ai channel.

SEO

Canonical URL, Open Graph and Twitter Card tags on every page
JSON-LD: WebSite, Person, BlogPosting, BreadcrumbList, FAQPage
Auto-generated sitemap-index.xml and rss.xml
IndexNow ping to Bing after each deploy

AIEO

/llms.txt: a site index written for LLMs
A .md version of every post — for example, this post in Markdown
A TL;DR at the top and an FAQ at the bottom, giving AI engines self-contained passages to quote
robots.txt explicitly allows AI search crawlers

FAQ

How is AIEO different from SEO?

SEO aims to rank in search results. AIEO (AI Engine Optimization, also called GEO) aims to make AI engines such as ChatGPT, Perplexity, Claude and Google AI Overviews understand your content and cite it with a link. They share the same foundations — good content, structured data and crawlability — but AIEO puts extra weight on answer-first writing, self-contained quotable paragraphs, llms.txt and Markdown versions of pages.

Why can't Google Analytics or Cloudflare Web Analytics see AI crawlers?

Both rely on JavaScript running in a browser. Crawlers like GPTBot, ClaudeBot and PerplexityBot fetch the HTML but never execute JavaScript, so they have to be logged on the server side — for example in a Cloudflare Worker that runs before the static assets, based on the User-Agent.

Does Cloudflare block AI crawlers by default?

Yes. Domains added to Cloudflare since 2025 block AI training crawlers by default and may have a managed robots.txt enabled. If you want to be cited by AI search, check the AI Crawl Control settings in the dashboard.