Hi! I'm Greg Lindahl, and I'm developing a new modular crawler called cocrawler.

If you've reached this page, you may have noticed my crawl activity in your logs, and you're wondering if I respect robots.txt. Well, I do, and if you'd like to block my crawler from crawling your site, you can use this:

User-agent: glindahl-cocrawler
Disallow: /

I crawl from the IP address range

Cocrawler has some algorithms to help it crawl politely. If you experience a problem, please let me know at lindahl (zat) pbm (zot) com. Thanks!