![]() |
|
Y! Search Crawler Updated To Slurp 3.0!
By: Navneet Kaushal 2008-04-17 Yahoo! announced the latest update to its Search Crawler called Yahoo Slurp 3.0. Although the Slurp 3.0 will recognize the same parameters and all robots.txt directives for Yahoo! Slurp... ...still it will be registered as Slurp 3.0 in the user web logs. As this new software is being slowly introduced, users would have an introduction to the following changes: The crawlers will originate from the crawl.yahoo.net.domain as usual, but they would start crawling from a different and much smaller set of IP addresses. As before, reverse DNS checks performed to identify Yahoo! crawlers will still be working. According to Yahoo!,Please note that if you're using IP-based recognition of our crawlers, you might see a drop in crawl/coverage from Yahoo! We strongly recommend that you move to 'reverse DNS based identification of Yahoo! Slurp if you're using any other method to avoid this problem. The current set of IPs will disappear from your web logs in the next several weeks. As mentioned above, the new user-agent Yahoo! Slurp 3.0 will follow existing robots.txt directives for 'Slurp' or 'Yahoo! Slurp. However, directives specific to Yahoo! Slurp 2.0 will not be recognized or followed by the new crawler. Therefore, it is recommended by Yahoo! that the user specifies the shorter version of User-agent: Slurp. These new changes will be only applicable to the main Yahoo! Web Search Crawlers and not would not affect specific search crawlers. For example, Yahoo! Slurp China and others. CommentsTag: Yahoo! Add to Del.icio.us | Digg | Reddit | Furl Have a bookmark! - About the Author: Nav is the founder and CEO of PageTraffic, a premier search engine company known for its assured SEO service, web design and development, copywriting and full time SEO professionals. Navneet has wide experience in natural search engine optimization, internet marketing and PPC campaigns. He is a prolific writer and his articles can be found in the "Best Articles" section of many websites and article banks. As a search engine analyst , he has over 9 years of experience and his knowledge is in application here. |
|
||||||||||||||||||||||||||||
| SearchNewz
is an iEntry, Inc. ® publication
©
$line) {
echo $line ;
}
?>
All Rights Reserved. Newsletter Archive - Privacy Policy - Legal - Sitemap - Contact Us - RSS Feeds - Newsletter Signup |