If you've reached this page, you may have noticed my crawl activity in your logs, and you're wondering if I respect robots.txt. Well, I do, and if you'd like to block my crawler from crawling your site, you can use this:
I crawl from the IP address range 18.104.22.168/28.
Cocrawler has some algorithms to help it crawl politely. If you experience a problem, please let me know at lindahl (zat) pbm (zot) com. Thanks!