An XML sitemap is most useful when it acts as a clean inventory of important canonical URLs. A large file full of redirects, errors, duplicates or noindexed pages creates noise and makes maintenance harder. The goal is not to list every address your server can produce; it is to list the URLs you want crawlers to treat as meaningful pages.
Include canonical, indexable URLs
Prefer the final HTTPS URL that your canonical tag and internal links use. Avoid listing old protocol variants, redirected paths, temporary test pages or duplicates that consolidate elsewhere.
Check HTTP status codes
Sample or crawl the sitemap URLs and look for 3xx redirects, 4xx errors and 5xx failures. A sitemap should not become a historical archive of removed pages. Remove dead URLs after the site has been updated to stop linking to them.
Use lastmod only when it reflects a meaningful change
A last-modified date is useful when it represents a substantive page update. Automatically setting every URL to today on every sitemap generation makes the field less informative and can hide which pages actually changed.
Keep sitemap entries aligned with canonicals
If a page’s canonical points to another URL, ask why the non-canonical version is still in the sitemap. Consistency across sitemap, canonical tags and internal links reduces ambiguity during crawling and auditing.
Validate XML structure and size
Malformed XML can prevent the file from being parsed. Validate the document, make sure URL values are properly escaped, and use sitemap index files if a site grows beyond a single sitemap’s supported limits.
Submit and monitor, but do not treat submission as guaranteed indexing
Submitting a sitemap can help discovery and monitoring, but search engines still decide whether to crawl and index individual pages. Use Search Console coverage and URL inspection alongside your own site checks.
Worked example
Example cleanup
If /tool-old redirects to /tool-new and /tool-new is the canonical live page, update internal links and keep only /tool-new in the sitemap. Then verify it returns 200 and is not noindexed.
Practical checklist
Before you finish
- List preferred canonical URLs only.
- Remove redirects and broken pages from the sitemap.
- Use accurate lastmod dates.
- Validate XML after automated generation.
- Re-audit the sitemap when templates or URL structures change.
Common mistakes
What to avoid
- Listing both canonical and duplicate parameter URLs.
- Keeping 404 URLs because they used to exist.
- Changing every lastmod date on every build.
- Assuming sitemap submission guarantees indexing.
Further reading from authoritative sources
Use these references when you need the underlying specification or platform guidance.