# See robot exclusion standard at http://www.robotstxt.org/ # General rules applying to everyone User-agent: * Disallow: /cgi-bin/ Disallow: /trap-bad-bots/