# # Following page has details on how to setup a robots.txt file and # related security issues: # # http://www.chami.com/tips/internet/010198I.html # # 1. Uncomment following two lines by removing "#" at the beginning to # indicate that all robots should stay away from the /cgi-bin/ folder #Blocked all searching 18-Feb-2008 User-agent: * Disallow: / #User-agent: * #Disallow: /cgi-bin/ #Disallow: /images/ #Disallow: /include/ #Disallow: /scripts/ # 2. Following two lines indicates that robots named "WWW_Robot_Name" # should stay away from this site. #User-agent: WWW_Robot_Name #Disallow: / # 3. Following two lines indicates that robots named "WWW_Wanderer_Name" # should not index files in the folder /private/ #User-agent: WWW_Wanderer_Name #Disallow: /private/