# robots.txt for Turboweb CMS User-agent: * Crawl-delay: 10 Disallow: /admin/ Disallow: /downloads/ Disallow: /video/ Disallow: /site/proof/ Disallow: /cart.php # Don't allow TurnitinBot User-agent: TurnitinBot Disallow: / # MJ12bot is crawling oddly http://mj12bot.com/ User-agent: MJ12bot Disallow: / User-agent: spbot Disallow: / Sitemap: https://www.gillrepairs.co.nz/sitemap.xml