April 26, 2003

Future Crawlers

Awhile back I remember Microsoft claiming that they were going to get into the search game. While I can't find the specific article off hand, I did notice this in my most recent robot access log:

131.107.163.46 - - [26/Apr/2003:20:54:49 -0400] "GET /robots.txt HTTP/1.1" 200 237 "-" "MicrosoftPrototypeCrawler (How's my crawling? mailto:newbiecrawler@hotmail.com)"

Looks like they are starting to make good on this promise. Now spammers do your thing...

Posted by Dan at April 26, 2003 06:05 PM
Comments