# This robots.txt file controls crawling of URLs under https://www.this-magazin.de User-agent: * Allow: / User-agent: SiteAuditBot Crawl-delay: 1 Allow: / User-agent: Semrushbot-SI Allow: / # Sitemap file Sitemap: https://www.this-magazin.de/sitemap.xml