Technical SEO Checklist for 2026: 30 Checks for Crawling, Indexing, Speed, and AI Search
A prioritized technical SEO audit checklist covering crawlability, indexing, site architecture, Core Web Vitals, JavaScript rendering, structured data, and AI crawler access, with the tools to check each item.
Key takeaways
- Technical SEO ensures search engines and AI crawlers can find, render, understand, and index your pages.
- Fix problems in order of impact: indexing blockers first, then architecture and duplication, then speed, then enhancements.
- Core Web Vitals "good" thresholds are LCP within 2.5 seconds, INP within 200 milliseconds, and CLS under 0.1.
- In 2026, check AI crawler access and server-side rendering as part of every audit.
Technical SEO is the work of making a website easy for search engines to crawl, render, index, and understand. It covers robots.txt and sitemaps, site architecture, canonical tags, page speed, mobile usability, JavaScript rendering, structured data, and security. Without it, even excellent content may never rank, or appear in AI-generated answers.
Use this checklist to run your own audit, or to check the work of an SEO agency. Items are grouped in priority order.
Tools you will need
- Google Search Console: indexing, Core Web Vitals, and search performance data
- Bing Webmaster Tools: Bing indexing, which also feeds some AI experiences
- PageSpeed Insights: lab and real-user performance data
- Rich Results Test and the Schema Markup Validator
- A crawler such as Screaming Frog SEO Spider, Sitebulb, or a similar tool
1. Crawlability (priority: critical)
- robots.txt is not blocking important sections. Check that CSS and JavaScript files are crawlable and that staging rules have not leaked into production.
- AI and search crawler access is intentional. Decide which bots to allow, such as Googlebot, Bingbot, OAI-SearchBot, ClaudeBot, and PerplexityBot. Confirm that firewall or bot-protection rules are not blocking legitimate crawlers. See our GEO guide.
- An XML sitemap exists, is referenced in robots.txt, contains only canonical 200-status URLs, and is submitted in Search Console and Bing Webmaster Tools.
- No important pages are orphaned. Every page you want ranked should be reachable through internal links.
- Server response is reliable. Check Search Console's Crawl Stats report for 5xx errors and slow response times.
2. Indexing (priority: critical)
- Key pages are indexed. Review the Page indexing report and inspect important URLs.
- No accidental noindex tags on templates, pagination, or new pages.
- Canonical tags are correct and self-referencing on unique pages, and point to the preferred version on duplicates.
- One version of the site resolves. HTTP, HTTPS, www, and non-www variants should redirect to a single preferred version.
- Thin and duplicate pages are consolidated, including tag archives, filtered URLs, and near-identical location pages. Online stores should also review faceted navigation.
3. Site architecture and internal linking (priority: high)
- Important pages sit within three clicks of the homepage.
- URLs are short, descriptive, and lowercase, using hyphens. For example: /services/digital-marketing.
- Topic clusters are linked together. Pillar pages link to supporting articles and back. This series links its digital marketing pillar page to each related guide.
- Anchor text is descriptive. Avoid "click here."
- Breadcrumbs are present and marked up with BreadcrumbList structured data.
4. Redirects and errors (priority: high)
- No redirect chains or loops. Each redirect should go straight to the final URL with a 301 or 308.
- Broken internal links (404s) are fixed or redirected to the most relevant page.
- Custom 404 pages return a real 404 status, not a soft 404 with a 200 code.
5. Page experience and Core Web Vitals (priority: high)
Google's Core Web Vitals measure real-user loading, responsiveness, and visual stability. Interaction to Next Paint (INP) replaced First Input Delay in March 2024.
| Metric | Measures | Good | Poor | Common fixes |
|---|---|---|---|---|
| Largest Contentful Paint (LCP) | Loading | 2.5 s or less | Over 4 s | Optimize hero images, use a CDN, reduce server response time, preload key resources |
| Interaction to Next Paint (INP) | Responsiveness | 200 ms or less | Over 500 ms | Reduce JavaScript, break up long tasks, defer third-party scripts |
| Cumulative Layout Shift (CLS) | Visual stability | 0.1 or less | Over 0.25 | Set image dimensions, reserve space for ads and embeds, avoid late-loading fonts shifting text |
- At least 75% of real-user visits pass each Core Web Vital, which is the threshold Google uses for assessment.
- Images are compressed, responsive, and in modern formats such as WebP or AVIF, with lazy loading below the fold.
- The site is fully usable on mobile. Google uses mobile-first indexing, so the mobile version is the one that is evaluated.
- HTTPS is enforced sitewide, with no mixed-content warnings.
6. JavaScript rendering (priority: high for modern frameworks)
- Core content and links are present in the initial HTML. Google can render JavaScript, but rendering can be delayed, and many AI crawlers do not execute JavaScript at all. Frameworks such as Next.js, Nuxt, and Astro support server-side rendering and static generation.
- Internal links use real anchor tags with href attributes, not click handlers.
- Metadata (title, description, canonical, and robots) is rendered server-side, not injected after load.
7. On-page technical elements (priority: medium)
- Every indexable page has a unique title tag of roughly 50 to 60 characters, and a unique meta description.
- Each page has one clear H1 and a logical H2 and H3 hierarchy.
- Images have descriptive alt text.
8. Structured data (priority: medium)
- Relevant schema.org markup is implemented and valid: Organization, WebSite, BreadcrumbList, Article or BlogPosting, Product, LocalBusiness, and Service, as relevant. The markup must match visible content. Note that since 2023 Google shows FAQ rich results mainly for well-known government and health sites, and has deprecated HowTo rich results. FAQ markup can still help machines understand your content, but do not expect FAQ snippets for a commercial site. See our schema markup guide.
9. International and multi-location (priority: if applicable)
- hreflang tags are correct and reciprocal for each language or region version, with an x-default where appropriate.
How to prioritize what you find
Score each issue by impact (how many important pages it affects, and how badly) and effort (developer time). Fix high-impact, low-effort items first. Indexing blockers always come before speed tweaks.
Example: An audit of a services website finds three issues: a staging noindex tag left on 40 service pages, a 4.1-second mobile LCP, and missing alt text on blog images. The noindex fix takes five minutes and unlocks rankings for the pages that generate leads, so it goes first. The LCP fix comes next because it affects every visitor. The alt text is a backlog task.
How often should you run a technical SEO audit?
Run a full audit at least twice a year, before and after any redesign or migration, and whenever organic traffic drops unexpectedly. Monitor Search Console weekly for new indexing or Core Web Vitals issues.
How to run the audit, step by step
- Collect the data. Run a full crawl with your crawler of choice, export Search Console's Pages (indexing) report and Core Web Vitals report, and pull your XML sitemaps. If you can, also get a sample of server logs to see how search engine bots actually crawl the site.
- Compare lists. Pages in the sitemap but not indexed, pages indexed but missing from the sitemap, and pages linked internally that return errors or redirects are the fastest way to find structural problems.
- Segment by template. Group issues by page type, such as product, category, blog, and location pages. Most technical problems are template-wide, so one fix in a template often resolves hundreds of URLs.
- Check rendering. Use the URL Inspection tool to view the rendered HTML for each important template and confirm that content, links, and metadata are present without relying on user interaction.
- Prioritize and document. Record each issue with the affected URLs, its impact, the recommended fix, and an owner, then work through them in priority order.
Common issues and how to fix them
| Issue | Typical cause | Fix |
|---|---|---|
| Important pages "Discovered, currently not indexed" | Weak internal linking, thin or duplicated content, or crawl budget spent on low-value URLs | Link to the pages from relevant hubs, improve content, and block or consolidate low-value URLs |
| Duplicate pages competing with each other | URL parameters, trailing slash or case variations, HTTP and HTTPS versions | Choose one canonical URL, redirect variants, and use consistent internal links |
| Redirect chains | Several migrations layered on top of each other | Point every old URL directly at its final destination and update internal links |
| Soft 404s | Empty category or search pages that return status 200 | Return a real 404 or 410, or add useful content |
| Slow Largest Contentful Paint | Large hero images, render-blocking scripts, slow server response | Compress and correctly size images, preload the hero image, defer non-critical JavaScript, and add caching |
| Layout shift | Images and ads without reserved dimensions, late-loading fonts | Set width and height or aspect ratios, reserve ad slots, and use font-display settings carefully |
| Content missing from rendered HTML | Client-side rendering that depends on user interaction | Server-render or statically generate primary content and links |
Technical SEO during a website migration
Migrations, whether a redesign, a domain change, or a platform switch, are where technical SEO matters most. Before launch, crawl the old site and build a complete redirect map from every old URL to its closest new equivalent. Keep titles, headings, and content of high-performing pages stable where possible. Test the new site on a staging environment that is blocked from indexing, then remove that block at launch. After launch, check that redirects return 301 status, submit the new sitemap, and monitor Search Console daily for coverage errors for several weeks.
Technical SEO for AI search
AI search systems rely on the same foundations: crawlable pages, fast responses, clean HTML, and accurate structured data. Check your robots.txt so that you are making a deliberate choice about which AI crawlers may access your content, and make sure important information is in the HTML rather than hidden behind scripts or images. Our generative engine optimization guide explains how to structure content so AI engines can cite it, and our schema markup guide covers the structured data types worth implementing.
Keeping technical SEO healthy between audits
- Add basic SEO checks to your release process: status codes, canonical tags, robots directives, and titles on key templates.
- Set up alerts for spikes in 404 or 5xx errors and sudden drops in indexed pages.
- Review Search Console's Pages and Core Web Vitals reports monthly.
- Recheck internal links whenever navigation or URL structures change.
- Re-crawl the site after every major release.
Technical SEO is one part of a wider program. For online stores, see the e-commerce SEO guide; for businesses serving specific areas, see the local SEO guide.
Crawl budget: when it matters and when it does not
Crawl budget is the number of URLs a search engine is willing and able to crawl on your site in a given period. For most sites with a few thousand pages, it is not a limiting factor. It becomes important for large e-commerce catalogues, marketplaces, news publishers, and sites that generate many URLs through filters, sorting, session IDs, or calendars.
If you suspect a crawl budget problem, look at server logs or Search Console's Crawl Stats report. Signs include bots spending most of their time on parameter URLs, important new pages taking weeks to be crawled, and large numbers of "Discovered, currently not indexed" URLs. Fixes include blocking low-value parameter patterns, consolidating duplicates with canonicals and redirects, improving server response times, and linking to priority pages more prominently.
A worked example: auditing a services website
A professional services firm with around 400 pages sees organic enquiries flatten. The crawl shows 60 internal links pointing to redirected URLs left over from an old site, 35 thin location pages with nearly identical text, and blog archive pages being indexed alongside the posts themselves. Search Console shows several service pages marked "Crawled, currently not indexed".
The team updates internal links to point at final URLs, merges the thin location pages into fewer, more useful regional pages, adds noindex to archive pages, and strengthens the service pages with clearer explanations, FAQs, and case examples. Over the following months, the service pages are indexed and begin to rank for commercial queries again. None of the fixes required new content volume; they removed the obstacles that were holding existing pages back.
Need help with technical SEO?
Technical SEO sits where marketing meets engineering, and that is how Vertex Cyber Tech is built. Our SEO team works alongside our web developers and cloud engineers, so we can audit and implement fixes in one engagement. Request a technical SEO audit.
Related guides: for practical next steps, read our Core Web Vitals and website speed guide.
Sources and further reading
Article FAQs
What is included in a technical SEO audit?
A technical SEO audit checks crawlability (robots.txt, sitemaps, internal links), indexing (noindex tags, canonicals, duplicates), site architecture, redirects and errors, Core Web Vitals and mobile usability, JavaScript rendering, on-page technical elements, structured data, and hreflang for international sites.
What are good Core Web Vitals scores?
Google considers Largest Contentful Paint of 2.5 seconds or less, Interaction to Next Paint of 200 milliseconds or less, and Cumulative Layout Shift of 0.1 or less to be good, measured at the 75th percentile of real-user visits.
How often should I do a technical SEO audit?
Run a full technical SEO audit at least twice a year, before and after any redesign or migration, and whenever organic traffic drops unexpectedly. Check Google Search Console weekly for new issues.
Does JavaScript hurt SEO?
JavaScript does not have to hurt SEO, but content that relies on client-side rendering can be indexed more slowly by Google and may be invisible to AI crawlers that do not execute JavaScript. Server-side rendering or static generation is the safest approach for important pages.
About Vertex Cyber Tech Digital Marketing Team
SEO strategists, content specialists, and web engineers at Vertex Cyber Tech Solutions who plan, build, and measure organic growth programs for businesses in India, the US, the UK, and beyond.
More articles