An XML sitemap lists a site’s URLs to guide search engine crawlers. Its role seems simple, but the quality of this file directly influences the speed and completeness of indexing. What differences in practice separate a well-maintained sitemap from a default-generated file that has never been audited?
XML Sitemap Tags: What Crawlers Actually Use
Most CMSs automatically generate a sitemap, with four tags per URL. Not all carry the same weight for Google.
See also : How to Successfully Create Your Professional Website with Cyber Vista
| Tag | Declared Role | Processing by Google |
|---|---|---|
| <loc> | Full URL of the page | Only mandatory tag, systematically used by Googlebot |
| <lastmod> | Date of last modification | Considered only if the date reflects a real change in content |
| <changefreq> | Estimated frequency of change | Ignored by Google for several years |
| <priority> | Relative priority among site pages | Ignored by Google |
Thus, two out of four tags have no effect on Google indexing. Keeping them poses no technical issue, but only <loc> and <lastmod> deserve real attention.
On Essentium’s homepage, we observe a segmented sitemap that illustrates this sorting logic by content type, a format that facilitates diagnosis in Google Search Console.
See also : Discover how to navigate easily with the Citizens News sitemap
Reliable Lastmod or Decorative Lastmod: The Gap That Changes Indexing
The <lastmod> tag is often misunderstood. Many sites update it automatically with each regeneration of the sitemap, even without any change to the page content. This practice renders the signal useless.
Google treats <lastmod> as a freshness signal as long as the date is honest. A page whose text, images, or structured data have not changed should not receive a new date.
The stakes are concrete: when Googlebot detects <lastmod> dates that are systematically recent without real change, it eventually ignores this tag for the entire site. The sitemap then loses its ability to signal real updates, those that justify a new quick crawl.
- Update <lastmod> only after a substantial change in content (revised text, new paragraph, added media)
- Do not confuse technical updates (template change, script addition) with content updates
- Regularly audit the consistency between <lastmod> dates and the actual modification dates of pages
A site that adheres to this discipline gives Googlebot an actionable signal. In contrast, a site that updates all its dates weekly without reason sends noise.
Segmenting Your XML Sitemap by Content Type
A single file listing all URLs works for a small site. Beyond a few hundred pages, segmentation by URL family becomes a management lever.
The principle is to create several sitemap files grouped in an index (sitemap index): one file for static pages, one for blog articles, one for product sheets, one for images, one for videos. Each file is declared in a sitemap index referenced by the robots.txt.
The interest goes beyond organization. In Google Search Console, indexing coverage can be filtered by submitted sitemap. If a file dedicated to product sheets shows an abnormally low indexing rate, the diagnosis is immediate: the problem lies within that URL family, not elsewhere.
This approach also allows for quick identification of orphan URLs or content that the CMS has generated without supervision (tag pages, date archives, parameterized URLs).
Canonical URLs and Excluded Pages: Cleaning the Sitemap for Better Indexing
A polluted XML sitemap with non-indexable URLs wastes crawl budget. Each URL in the sitemap must return a 200 status and be canonical.
URLs to systematically exclude:
- Pages with 301 or 302 redirects (the destination URL is sufficient)
- Pages with a
noindextag (direct contradiction with presence in the sitemap) - Parameterized URLs that duplicate existing content (filters, sorts, non-canonical paginations)
- Low-value pages generated automatically (date archives, empty tag pages)
Google specifies in its documentation: a sitemap should only contain URLs that the site wants to see indexed. Including blocked or redirected URLs does not prevent the indexing of the right pages, but muddles the signals sent to crawlers and complicates tracking in Search Console.
A quarterly audit of the sitemap, cross-referenced with coverage reports, allows for the detection of inconsistencies before they accumulate. Crawl tools replicate Googlebot’s path and highlight discrepancies between the declared sitemap and the actual state of the URLs.
Submitting and Declaring the XML Sitemap to Search Engines
Generating a clean sitemap is not enough. It must be declared so that search engines can find it without relying on internal linking.
Two declaration methods coexist. The first goes through the robots.txt file, where a line Sitemap: https://example.com/sitemap.xml indicates the file’s location to any robot that consults the robots.txt. The second goes through direct submission in Google Search Console or Bing Webmaster Tools, which also allows for tracking processing errors.
Declaring the sitemap in robots.txt and in Search Console remains the recommended combination. The robots.txt covers all engines, while Search Console provides detailed feedback on discovered, indexed, or excluded URLs.
After each major site update (migration, restructuring, massive page addition), a new submission in Search Console accelerates the consideration of changes.
The XML sitemap does not improve a page’s ranking. It ensures that pages deserving to be indexed are indeed discovered, within a reasonable timeframe, with reliable metadata. The difference lies in the rigor of maintenance, not in the mere existence of the file.