Fix sitemap indexing consistency

[profilesearch]

Summary

• remove one duplicate URL from the public sitemap
• remove an intentionally noindex article from the sitemap while leaving the page unchanged
• add non-vacuous regression coverage for unique sitemap URLs and sitemap/indexing consistency
• parse the sitemap as XML and robots metadata as HTML instead of relying on regular expressions

Search Console finding

The current robots policy only disallows /cgi-bin/. A read-only Search Console URL Inspection audit covered every unique live sitemap URL and found zero sitemapped URLs blocked by robots.txt. The email alert is therefore consistent with intentional blocking of invalid /cgi-bin/ paths, not a production-content crawl defect. robots.txt is intentionally unchanged.

Test plan

• Targeted RED/GREEN regression tests
• 390 backend tests passed
• 81 browser tests passed; 5 skipped
• Security static checks passed
• Public-shell checks passed
• Performance budgets passed
• Dependency audit found no known vulnerabilities
• Sitemap XML parsed successfully
• Local sitemap has 137 entries and 137 unique URLs
• git diff --check passed

Risk

Low. The production change only removes two <url> blocks from sitemap.xml: one duplicate and one URL whose HTML already opts out of indexing. No page content, routing, robots policy, API, payments, or user data changes.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
gohire-humans	Ready	Preview, Comment	Jul 19, 2026 2:57am