Skip to content
search-engine-optimizedby Netsolution

What sitemaps actually do

A sitemap points addresses out to an engine, without guaranteeing they will be crawled or indexed. Two of its optional tags, change frequency and priority, are explicitly ignored by the engine's vendor. The last modification date is used only if it can be verified.

What is a sitemap for, exactly?

For telling an engine that addresses exist. Nothing more, and that is already useful where discovery through links works badly: a new site nobody points at yet, a catalogue where some entries are only reachable after three filters, an archive the navigation no longer shows.

What it does not do is written in the documentation, and deserves reading before spending half a day building one.

Providing a sitemap does not guarantee that all the items it contains will be crawled and indexed.

officialSource : Google Search Central, “What Is a Sitemap”

Pointing out is not obtaining. A sitemap puts an address in front of a crawler; the decision to crawl, and then the decision to index, remain entirely intact and are taken elsewhere. It is the same distinction as everywhere else: you act on what is offered, never on what is kept.

The two tags nobody reads

The protocol defines three optional tags inside each address: last modification date, change frequency, priority. Of those three, two do nothing at all, and that is not a deduction.

The engine's vendor writes that it ignores the values of the priority and change frequency tags.

officialSource : Google Search Central, “Build and submit a sitemap”

The sentence is short and carries no reservation. It is worth setting alongside what the protocol itself announced, long before:

The protocol itself writes that the priority assigned to a page is not likely to influence the position of addresses in the results pages, and that it only distinguishes addresses within the same site.

officialSource : sitemaps.org, protocol version 0.9

The protocol calls change frequency a hint rather than a command: a crawler may visit less often than the value announced, or more often.

officialSource : sitemaps.org, protocol version 0.9

In other words, the standard that created those two fields already warned they were only a suggestion, and the engine’s vendor eventually wrote that it does not look at them. In between, a generation of plugins and generators set about filling them in carefully, and a whole literature explained how to tune them properly.

Set the priority of your important pages to 1.0 and secondary pages to 0.3, then switch product pages to a daily change frequency: the engine will crawl your strategic pages first.

Both values are ignored, and the engine's vendor writes so in its documentation. The advice asks for real work, produces a heavier file, and has no effect — the worst of the three possible outcomes.

The only optional tag that counts, and what destroys it

That leaves the last modification date. It is used, but conditionally, and the condition is the interesting part.

The last modification date is used if it is consistently and verifiably accurate, for instance by comparing it with the page's actual last modification.

officialSource : Google Search Central, “Build and submit a sitemap”

“Consistently and verifiably”: the field is not taken on trust, it is checked against the page. A site whose dates correspond to real changes supplies usable information. A site where every date moves together at each publication supplies noise, and the field loses what made it interesting.

An automatically produced sitemap that stamps today's date on every address at each build makes unverifiable the one optional field the vendor says it uses.

hypothesis

The status of that last sentence is deliberate. The documentation says on what condition the date is used; it does not say what happens when the condition is not met, and we are not going to invent it. What is certain is that the condition is no longer met — the rest is a deduction, displayed as one.

The practical gesture fits on one line: check what your generator produces. If every address carries the same date and it changes at every build, the field is lying.

The limits of the format, and what they imply

A sitemap file is limited to fifty thousand addresses and fifty uncompressed megabytes, whatever its form; beyond that it has to be split and an index file declared.

officialSource : Google Search Central, “Build and submit a sitemap”

That split is purely mechanical. A site divided into twelve files declared by an index is treated neither better nor worse than a site that fits in one: it is simply conformant to the format. No arrangement of the files amounts to a strategy, and the idea of grouping addresses by theme to “guide” crawling rests on nothing published.

Do you need one?

The engine's vendor writes that a sitemap is not needed for a site of about five hundred pages or fewer whose pages are properly linked to one another.

officialSource : Google Search Central, “What Is a Sitemap”

The answer therefore depends on size and above all on internal linking. A small, well-linked site gets nothing from one, and blaming it for that amounts to asking it to solve a problem it does not have. A large catalogue, a new site with no inbound links, a body of media content: there, a sitemap works.

Publishing a sitemap and letting the site itself produce it

hypothesis

Effort
low
Time to effect
months
Risk
low
Durability
structural

The effort is low on one condition: that the file is produced at build time, by the site. Kept by hand, it drifts from the real site within weeks, and a sitemap that lies costs more than no sitemap at all.

The time-to-effect axis is not a waiting period but an absence of immediate effect: the file points things out, it triggers nothing. It is nonetheless one of the few gestures that makes sense on a brand new site, before there is a single inbound link.

The risk axis deserves a pause. It does not come from the file itself but from what you declare in it: addresses that redirect, that answer with an error, or that name another address as canonical. The sitemap then becomes a list of contradictions, and it says more about the disorder of the site than about its pages.

What does not work, or no longer works

Three practices survive well past the point where they stopped making sense.

Setting priority and change frequency, first, handled above. Then submitting the file several times a day in the hope of speeding something up: submission declares that the file exists, it does not order a visit. Finally, declaring every address on the site without distinction, including those that redirect, those that answer with an error and those that point at another canonical address — a sitemap is only useful if it describes pages as they are actually served.

The check that settles all three is the same, and it takes two minutes: open your own sitemap, pick a dozen addresses at random, and verify that each answers directly, without a redirect, and names itself as its own canonical address.

Sources

  1. What Is a Sitemap — Google Search Centralaccessed on
  2. Build and submit a sitemap — Google Search Centralaccessed on
  3. Sitemap Protocol, version 0.9 — sitemaps.orgaccessed on

Block revalidated on

Frequently asked questions

Does a sitemap get the pages it contains indexed?
No. The documentation writes that it does not guarantee that all the items it contains will be crawled and indexed. It points addresses out, which helps when they are poorly linked to one another; it never replaces internal links, and exerts no pressure on the decision to index.
Should you fill in priority and change frequency?
Both tags are optional in the protocol, and the engine's vendor writes that it ignores their values. The protocol itself already warned that priority would not influence the position of addresses. Filling them in is not an error; it is work whose result is nil.
Is the last modification date good for anything?
Yes, on one written condition: it is used if it is consistently and verifiably accurate, by comparison with the page's actual last modification. A date invented or reset at every build therefore strips the field of what gave it its value.
My site is small: do I need a sitemap?
The documentation answers no for a site of about five hundred pages or fewer whose pages are properly linked. A sitemap becomes useful when the site is large, when it is new and receives few external links, or when it carries a lot of media content.
What happens when the file grows too big?
The limit is fifty thousand addresses and fifty uncompressed megabytes per file. Beyond that the file has to be split, and an index file can declare the set. That split is mechanical: it changes nothing about how the addresses are treated.