SEO · MOFU
Fix Your Technical SEO Before You Chase AEO
Devansh Trivedi — 28 September 2026 · 8 min read
The expensive mistake
You cannot optimize a page for an answer engine that cannot reliably reach, read, or trust it.
Most founders I talk to this year want their business to appear when someone asks ChatGPT or Perplexity for a recommendation. They have read about AEO and GEO. Some have already paid for an "AI visibility audit."
Then I open their Google Search Console. Half their pages are not indexed. The sitemap lists URLs returning 404 errors. Service pages take five seconds to load on a phone. GA4 records page views but not one form submission.
AEO is not the next task. The technical base is the task.
Why does technical SEO still matter if people search in ChatGPT?
Because every AI citation begins with a page that a machine could discover, retrieve, understand, and trust.
AI answers are built from pages that crawlers could reach and read. Google's AI Overviews and AI Mode draw from Google's own index. ChatGPT, Claude, and Perplexity run their own crawlers and also lean on traditional search indexes.
- Step one
- Reach
- Step two
- Read
- Step three
- Trust
- Step four
- Cite
The crawler can request the URL without a block, loop, or server error.
The useful content exists in the HTML the crawler actually receives.
Canonical, internal-link, and structured-data signals agree on what the page is.
Only then can an answer engine consider the page as a source.
Is your site healthy enough to be crawled?
A crawler-blocking error matters more than a hundred cosmetic warnings. Fix access, status codes, and broken paths first.
Start with a site health audit. Semrush Site Audit, Ahrefs Site Audit, or Screaming Frog will each give you a list of errors within an hour.
Fix in this order
- 01
Pages that fail
Resolve 4xx and 5xx responses on URLs that users or crawlers still reach.
- 02
Paths that waste the crawl
Remove redirect loops and make internal links point directly to the final URL.
- 03
Reports that inflate confidence
Treat the health score as triage, not success. A score in the 90s does not guarantee rankings; a score in the 60s usually means something important is broken.
Related reading: SEO for Startups: What to Do Before You Publish Your First Blog Post
How many of your pages are actually indexed?
Count the pages you need in search, then compare that number with what Google has actually kept. The gap is your real starting point.
Open the Page indexing report in Google Search Console. Compare the number of indexed pages against the number of pages you want in search.
Discovered, currently not indexed
Google knows the URL exists but has not crawled it. Weak internal linking, crawl waste, or low perceived importance is usually the place to look.
Crawled, currently not indexed
Google read the page and chose not to keep it. Investigate thin, duplicate, or low-value content.
Use the URL Inspection tool on your most important pages. It shows you whether the page is indexed, which canonical Google picked, and when it was last crawled.
Are your canonical tags pointing to the right pages?
Every indexable page should name one preferred, final URL—and your links, sitemap, redirects, and canonical tag should all name the same one.
A canonical tag tells search engines which version of a page is the original. Most small sites have more duplicate URLs than their owners realize. The same page can exist with and without "www," with and without a trailing slash, over HTTP and HTTPS, and with tracking parameters attached.
Is Google wasting crawl budget on the wrong pages?
Probably not on a small brochure site. On stores, listings, filters, calendars, and internal search pages, it can quietly consume the crawl meant for pages that make money.
Crawl budget is the number of URLs a search engine is willing to crawl on your site in a given period. For a site with a few hundred pages, it is rarely the bottleneck. For e-commerce stores, listing sites, and anything with filters or search pages, it matters a lot.
Find and cut the waste
- 01
Read Crawl stats
Check which file types, response codes, and URL groups receive the most requests.
- 02
Identify infinite spaces
Look for filter combinations, internal search results, calendar pages, parameter URLs, and redirects.
- 03
Remove the invitation
Block low-value patterns where appropriate, fix redirect chains, and stop linking to parameter URLs from templates. AI crawlers often have tighter limits than Googlebot.
Do your sitemap and robots.txt match what you want found?
Your sitemap should contain only the URLs you want indexed. Your robots.txt should block only what you do not want crawled. Those are related jobs, not the same job.
sitemap.xml
List only canonical, indexable URLs returning a 200 status. Remove redirects, noindexed pages, and 404s. The discovered count should roughly match the pages you genuinely want found.
robots.txt
Read it line by line. It controls crawling, not indexing. A blocked URL can still appear in results, and blocking CSS or JavaScript can stop crawlers from understanding a page.
I still find live sites with Disallow: / left over from staging. This is also where you decide how to treat GPTBot, OAI-SearchBot, ClaudeBot, and PerplexityBot. If you want to be cited, check that a security plugin or CDN rule is not turning them away.
Can Google and AI crawlers read content that loads through JavaScript?
Google can render JavaScript. Many AI crawlers do far less rendering. If your useful content exists only after scripts run, the page may look complete to a person and empty to an answer engine.
Googlebot renders JavaScript. It loads your page in a headless browser, runs the scripts, and indexes the finished result. So a site built on React or a similar framework can rank well on Google even if its content only appears after the scripts run.
Most AI crawlers do not do this. OpenAI, Anthropic, and Perplexity each document the crawlers they use to discover and retrieve web content. If your content only exists after JavaScript runs, those crawlers may see an empty page.
Test this yourself. Right-click your page and choose "View page source." This is not the same as "Inspect." If your headings, body copy, prices, and FAQs are missing from the source, AI crawlers cannot read them.
Same URL, two very different pages
Bookkeeping for growing agencies
Monthly books closed by the fifth, reconciled by a real accountant, with a dashboard your team actually reads.
Are your meta titles and descriptions doing their job?
A unique title and description will not rescue a weak page, but missing or duplicated metadata makes a strong page harder to interpret and harder to choose.
Every indexable page needs a unique title tag and meta description. Titles work best at around 50 to 60 characters, with the main topic near the start. Descriptions work best at around 150 to 160 characters and should say what the page answers.
- Title
- 50–60
- Description
- 150–160
Characters is a useful editing range, not a hard ranking rule. Put the main topic near the start.
Characters is a practical range. State what the page answers instead of writing empty brand language.
Do your pages pass Core Web Vitals?
Use field data from real visits: LCP at 2.5 seconds or less, INP at 200 milliseconds or less, and CLS at 0.1 or less.
Core Web Vitals measure real page experience. Check the report in Search Console for field data. PageSpeed Insights is useful for diagnosis, but its lab score is a simulation. Your real users' numbers matter more.
Check your own numbers
Drag each marker or type your score to see where you land against Google's thresholds.
Use the field data from the Core Web Vitals report in Search Console, not your PageSpeed lab score.
The usual repair order
- 01
Fix image weight
Serve WebP or AVIF, size images for their rendered space, and declare width and height so the layout does not jump.
- 02
Protect the LCP image
Lazy-load images below the fold, but not the main image that usually determines Largest Contentful Paint.
- 03
Interrogate third-party scripts
Chat widgets, heatmaps, consent tools, and old tracking tags can slow every interaction on the page.
Is your cache serving old versions of your pages?
It can. A corrected title, price, or FAQ is not fixed for search until the cached HTML stops serving the old version.
Are GSC and GA4 tracking what matters?
Track the actions that make money, not just page views. Otherwise you can report more visibility while remaining blind to whether it created a lead or sale.
Make the measurement trustworthy
- 01
Connect discovery to behavior
Set up Search Console as a Domain property and link it to GA4.
- 02
Define the money events
Mark form submissions, booking confirmations, phone clicks, purchases, and demo requests as key events. Test each one in GA4 DebugView.
- 03
Clean the signal
Filter internal traffic, verify the consent banner is not suppressing most measurement, and separate referrals from sources such as chatgpt.com and perplexity.ai.
Related reading: Building an SEO Function From Scratch at a Fintech Startup
Where is your conversion funnel leaking?
Find the step with the steepest drop, use it yourself on a phone, and fix it before buying more traffic.
Map the path in a GA4 funnel exploration. A typical service journey runs from landing page to service page to contact page to form submission. The biggest loss often reveals a silent form failure, broken booking embed, missing confirmation, or weak next action.
Funnel leak finder
Drop in your own numbers from a GA4 funnel exploration. Each step should be equal to or smaller than the one before it. The biggest leak is highlighted for you.
Landing page visits
36% continue · 64% drop off
Service page views
22% continue · 78% drop off
Contact page views
9.0% continue · 91% drop off
Form submissions
Your biggest leak: Contact page views to Form submissions (91% drop). Fix this before you buy more traffic.
When does AEO or GEO become worth the effort?
When your important pages are crawlable, indexed, fast, readable without client-side rendering, and tied to measurable outcomes.
FAQ
Do I need technical SEO before AEO?
Yes. AI answer engines cite pages their crawlers can reach and read. If your pages are blocked, unindexed, slow, or empty in raw HTML, AEO work has nothing to build on.
Can ChatGPT read JavaScript-rendered content?
Mostly no. ChatGPT's and Claude's crawlers download JavaScript files but do not run them. Content that only appears after JavaScript runs is invisible to them.
What are the Core Web Vitals targets?
LCP of 2.5 seconds or less, INP of 200 milliseconds or less, and CLS of 0.1 or less, measured on real user visits.
Does robots.txt remove a page from Google?
No. robots.txt controls crawling, not indexing. Use a noindex tag to keep a page out of search results.
How do I track AI search traffic in GA4?
Build a custom channel group or exploration that isolates referrals from domains like chatgpt.com and perplexity.ai.
Go deeper
Want a second pair of eyes on your site?
I audit the technical base first, then build the content and AEO work on top of it.
Read next
