For Programmers: Free Programming Magazines  


Home > Archive > PHP Mirrors > November 2004 > cvs: php-news-web / robots.txt









You are viewing an archived Text-only version of the thread. To view this thread in it's original format and/or if you want to reply to this thread please [click here]

 

Author cvs: php-news-web / robots.txt
Derick Rethans

2004-11-26, 9:01 am

derick Fri Nov 26 07:16:26 2004 EDT

Added files:
/php-news-web robots.txt
Log:
- Added a robots.txt file so that we stop some crawlers from bringing down
the NNTP server.



http://cvs.php.net/co.php/php-news-...s.txt?r=1.1&p=1
Index: php-news-web/robots.txt
+++ php-news-web/robots.txt
User-agent: *
Disallow: /
Sponsored Links







Also available: Server administration forum archive | Web Design forum archive | Software forum archive | Hardware reviews archive

Copyright 2008 codecomments.com