Resources · 108
Sitemaps: verify URLs, shards and lastmod dates
Check the file offered to crawlers beyond simple XML syntax validity.
· 2 min
What this guide helps achieve
- Define expected URLs
- Check contents and responses
- Prepare partitioning
- Make lastmod meaningful
- Track discrepancies
Quick check
- Does a sitemap guarantee indexing?
- Should lastmod change on every generation?
Step-by-step method
- 01
Define expected URLs
Start from published pages and canonical destinations. Exclude search filters, errors and redirects from the expected set; compare languages and families separately.
Deliverable: eligible URL set.
- 02
Check contents and responses
Check absolute addresses, encoding, escaped characters and final responses. Separate XML validity, HTTP availability and editorial relevance; each needs a check.
Deliverable: structure and response checks.
- 03
Prepare partitioning
Measure URL count and uncompressed size. The protocol defines file limits; establish stable partitioning and an index before excessive generation becomes unusable.
Deliverable: tracked limits and shards.
- 04
Make lastmod meaningful
Link dates to significant content changes, not each request. Check dependencies across translations, articles and data; invalidate a cache when its source changes.
Deliverable: date provenance.
- 05
Track discrepancies
Compare expected and emitted URLs, duplicates and inaccessible shards. Search reports describe discovery and processing; submission does not guarantee indexing.
Deliverable: discrepancy report and owner.
Decide what enters the sitemap
Check the observed state before choosing next steps.
Eligible published page
Emit its final URL and justified change date.
Filter, error or redirect
Exclude it from the landing-page set.
Oversized shard
Partition and check files and their index.
Reusable worksheet
Complete with your authorised observations. These fields are a working template, not observed results.
| Field | Information to record |
|---|---|
| URL | Canonical, language and published state |
| Shard | Uncompressed bytes and count |
| Date | Change source and check |
Worked example
Illustrative situation
Illustrative case: a sitemap includes every article but misses a new translated-guide family. Its XML remains valid.
Decision and expected evidence
Compare the expected set by family and language; fix generation dependencies and then inspect served files.
Distinguish the mechanisms
| Mechanism | Purpose | Check or limitation |
|---|---|---|
| Valid XML | Interpretable document | Can contain inappropriate URLs |
| Sitemap index | Reference multiple files | Check every shard |
| lastmod | Significant change date | Avoid artificially renewed dates |
Management indicators
| Indicator | What it measures | First action |
|---|---|---|
| Coverage | Eligible pages actually emitted | Address omissions |
| Quality | Duplicates and non-final destinations | Correct generation |
Common pitfalls
- Renew all dates without content changes
- Include redirects as primary destinations
- Check only compressed size
Frequently asked questions
Does a sitemap guarantee indexing?
No. It helps discovery. Availability, directives and content quality remain separate.
Should lastmod change on every generation?
Keep a date tied to the affected content. Technical regeneration without editorial change is insufficient.
Official references
References consulted: . The method and worksheet propose checks to adapt to your context; they do not constitute certification.
Continue exploring
Related analysis
