Technical
The most documented part of the field, and the one where intuition misleads most often. Every guide here starts from a document published by a search engine, and several of them exist only because that document says the opposite of what circulates.
Where to come in, and why that order
Through indexing, never through speed. That is the opposite of the spontaneous order: speed is measured in one click, it produces a figure, it gives the feeling of progress. Indexing asks you to understand a mechanism and produces no score at all.
Yet the order is forced by the mechanism itself. A page the engine never downloaded will benefit from no loading optimisation, and neither will a page it downloaded but did not keep. Working on the speed of a page missing from the index is repainting a façade without having checked that the house is connected.
The mistake that costs most here
Confusing what the site exposes with what the engine kept. A crawler walks your pages and hands back an inventory; the engine's console reports what it actually kept. The two lists overlap without ever coinciding, and spending a long time fixing pages the engine was not concerned with is the most frequent outcome of a badly framed technical audit.
The second mistake, particular to this area: piling signals up. A redirect on top of a canonical on top of an indexing instruction, because nothing has moved in three days, produces a configuration nobody can read any more. The documentation says the opposite — make the pages different enough, then wait.
What this section brings that others do not
Every guide here starts from a document published by a search engine, cited with the date it was read. Several exist only because that document says the opposite of what circulates: crawl budget does not concern sites of a few hundred pages, a page disallowed in the robots.txt file can stay visible in the results, and a declared canonical obliges the engine to nothing.
It is also the section most exposed to tools selling a technical score. No engine publishes one, and nothing connects a list of conventional deviations to a change of position — which the crawl and audit tool directory details tool by tool, with what each one does not do. That directory is published in French.
What it does not cover
Hosting, handled as a tool entry because it is bought: it acts on server response time, and on nothing else in search. Building a site, which is not this site's subject. And complex multi-domain migrations, of which the migration guide handles only the common case — one domain, one address plan, one switchover.
The guides in this section
- A/B testing and search
The four published rules that keep an interface test from damaging search: do not serve two versions by requester, canonical rather than noindex, temporary redirect, and a test that ends.
- AI crawlers and Google-Extended
What the documentation writes about the Google-Extended token, about GoogleOther, about the absence of a separate user agent, and about why blocking that token does not remove you from AI answers in Search.
- Asking for a recrawl
The two documented ways to flag a new or changed page, the quotas around them, and the sentences by which the documentation refuses to guarantee anything.
- Breadcrumbs and multiple paths
What the documentation writes about breadcrumb markup, about declaring several trails for the same page, and about the access conditions that make it inoperative.
- Canonicals, redirects and duplication
The documented methods for naming a canonical address, the difference between a permanent and a temporary redirect, and what happens when the engine does not follow your declaration.
- Controlling what is shown of your page
The published rules that limit the snippet, the image preview and the use of your text by generative answers: nosnippet, data-nosnippet, max-snippet, max-image-preview, and the conflict rule.
- Crawl budget
What the documentation calls crawl capacity limit and crawl demand, the three site profiles concerned, and why the great majority of sites have no business worrying about it.
- Error pages and the soft 404
What the published documentation writes about 4xx and 5xx codes, about an address leaving the index, about the soft 404, and about the error page itself, which the rater guidelines score like any other page.
- How robots.txt is interpreted
The interpretation rules published by the engine's vendor: recognised fields, arbitration between contradictory rules, maximum size, caching, response codes. And why blocking crawling does not remove an address from the index.
- Infinite scroll and lazy loading
What the documentation asks so that content loaded on demand is seen: loading tied to the viewport rather than to a gesture, a stable address per chunk, and one check on the rendered page.
- JavaScript and engine-side rendering
The three phases the documentation describes for a JavaScript page, the delay of the render queue, and the precise places where content produced by a script disappears.
- Marking up a forum and its discussions
What the documentation expects of discussion markup: replies nested as the thread organises them, later pages tied back to the main address, and the declaration a listing page must not make.
- Migrating a site without losing its addresses
The documented steps of an address change: permanent server-side redirects, how long to keep them, internal links, sitemap. And what the engine's vendor says about the period that follows.
- Multilingual sites and hreflang
The four documented address structures with their trade-offs, declaring language versions, and two explicit discouragements: IP address analysis and automatically redirecting a visitor from one version to another.
- Pagination and faceted navigation
What the documentation says about numbered pages and filters: real links, a proper address per page, a canonical per page, the parameter separator, and why filter combinations cost crawling.
- Pausing a website
Temporary closure, out of stock, holidays: the recommended method, the moves explicitly advised against, and the one situation where taking the site down is defensible, for a day or two.
- Paywalled content and sampling
What the documentation asks for so that subscriber-only content can be indexed without being mistaken for cloaking: the markup to add, the two forms of sampling described, and the published numbers.
- Removing a page from the results
What the documentation writes about the removals tool and how long it lasts, about the ways to make a removal permanent, and about why robots.txt does not remove a page.
- Security issues and browser warnings
What the documentation files under malware, unwanted software and social engineering, the difference between the two labels in the report, and the rule that makes a site answerable for the ads it shows.
- Speed and loading experience
The three metrics the browser vendor publishes, their thresholds at the 75th percentile, and the sentence that puts speed back in its place: relevance comes first, even when the experience is poor.
- Structured data and rich results
What structured data describes, the rule forbidding markup of anything the reader cannot see, and why correct markup guarantees no rich display at all.
- Verifying that a crawler is Googlebot
The double DNS lookup the documentation describes, the three families of crawlers and their suffixes, the published address lists, and what a user agent name cannot establish.
- What sitemaps actually do
What a sitemap really brings, what it does not guarantee, the size limits of the protocol, and the exact fate of the change frequency and priority tags at the hands of the engine's vendor.
- Why a page is not indexed
The states the engine publishes itself to explain a page missing from the index, how to read them in order, and the difference between a blocked page and a discarded one.
Sources
- AI features and your website — Google Search Centralaccessed on
- Ask Google to recrawl your URLs — Google Search Centralaccessed on
- Block Search indexing with noindex — Google Search Centralaccessed on
- Breadcrumb (BreadcrumbList) structured data — Google Search Centralaccessed on
- Build and submit a sitemap — Google Search Centralaccessed on
- Discussion forum (DiscussionForumPosting) structured data — Google Search Centralaccessed on
- Fix lazy-loaded content — Google Search Centralaccessed on
- Flexible Sampling general guidance — Google Search Centralaccessed on
- General Guidelines, version of 11 September 2025, 182 pages — Googleaccessed on
- General structured data guidelines — Google Search Centralaccessed on
- Google Search spam policies — Google Search Centralaccessed on
- Google's common crawlers — Googleaccessed on
- How Google interprets the robots.txt specification — Google Search Centralaccessed on
- How HTTP status codes, and network and DNS errors, affect Google Search — Google Search Centralaccessed on
- How to move a site — Google Search Centralaccessed on
- How to specify a canonical URL with rel="canonical" and other methods — Google Search Centralaccessed on
- In-depth guide to how Google Search works — Google Search Centralaccessed on
- IndexNow — frequently asked questions — IndexNow.orgaccessed on
- Interaction to Next Paint (INP) — web.dev (Chrome team)accessed on
- Introduction to robots.txt — Google Search Centralaccessed on
- Malware and unwanted software — Google Search Centralaccessed on
- Managing crawling of faceted navigation URLs — Google Search Centralaccessed on
- Managing multi-regional and multilingual sites — Google Search Centralaccessed on
- Minimize A/B testing impact in Google Search — Google Search Centralaccessed on
- Optimize your crawl budget — Google Search Centralaccessed on
- Page indexing report — Google Search Consoleaccessed on
- Pagination, incremental page loading, and Search — Google Search Centralaccessed on
- Provide a site name to Google Search — Google Search Centralaccessed on
- Redirects and Google Search — Google Search Centralaccessed on
- Remove a page hosted on your site from Google — Google Search Centralaccessed on
- RFC 9309 — Robots Exclusion Protocol — IETFaccessed on
- Robots meta tag, data-nosnippet, and X-Robots-Tag specifications — Google Search Centralaccessed on
- schema.org (version 30.0, published on 2026-03-19) — schema.orgaccessed on
- Sitemap Protocol, version 0.9 — sitemaps.orgaccessed on
- Social engineering (phishing and deceptive sites) — Google Search Centralaccessed on
- Structured data for subscription and paywalled content — Google Search Centralaccessed on
- Structured data markup that Google Search supports — Google Search Centralaccessed on
- Tell Google about localized versions of your page — Google Search Centralaccessed on
- Temporarily pause or disable a website — Google Search Centralaccessed on
- Understand the JavaScript SEO basics — Google Search Centralaccessed on
- Understanding page experience in Google Search results — Google Search Centralaccessed on
- Verify requests from Google crawlers and fetchers — Googleaccessed on
- Web Vitals — web.dev (Chrome team)accessed on
- What Is a Sitemap — Google Search Centralaccessed on
Block revalidated on