# www.segmentseo.com robots.txt - do your thing, crawlers User-agent: * Disallow: /blog-categories/* Disallow: /search Sitemap: https://www.segmentseo.com/sitemap.xml