iEntry 10th Anniversary RSS Contact


Y! Search Crawler Updated To Slurp 3.0!

By: Navneet Kaushal
2008-04-17

Yahoo! announced the latest update to its Search Crawler called Yahoo Slurp 3.0. Although the Slurp 3.0 will recognize the same parameters and all robots.txt directives for Yahoo! Slurp...

...still it will be registered as Slurp 3.0 in the user web logs.

As this new software is being slowly introduced, users would have an introduction to the following changes:

The crawlers will originate from the crawl.yahoo.net.domain as usual, but they would start crawling from a different and much smaller set of IP addresses. As before, reverse DNS checks performed to identify Yahoo! crawlers will still be working. According to Yahoo!,Please note that if you're using IP-based recognition of our crawlers, you might see a drop in crawl/coverage from Yahoo! We strongly recommend that you move to 'reverse DNS based identification of Yahoo! Slurp if you're using any other method to avoid this problem. The current set of IPs will disappear from your web logs in the next several weeks.

As mentioned above, the new user-agent Yahoo! Slurp 3.0 will follow existing robots.txt directives for 'Slurp' or 'Yahoo! Slurp. However, directives specific to Yahoo! Slurp 2.0 will not be recognized or followed by the new crawler. Therefore, it is recommended by Yahoo! that the user specifies the shorter version of User-agent: Slurp.

These new changes will be only applicable to the main Yahoo! Web Search Crawlers and not would not affect specific search crawlers. For example, Yahoo! Slurp China and others.

Comments

Tag:

Add to Del.icio.us | Digg | Reddit | Furl

Have a bookmark! -


About the Author:
Nav is the founder and CEO of PageTraffic, a premier search engine company known for its assured SEO service, web design and development, copywriting and full time SEO professionals.

Navneet has wide experience in natural search engine optimization, internet marketing and PPC campaigns. He is a prolific writer and his articles can be found in the "Best Articles" section of many websites and article banks. As a search engine analyst , he has over 9 years of experience and his knowledge is in application here.


Visit the SearchNewz Directory
Do you have a search site?
Submit it free to the internet's best search industry directory. » Click Here
Search Engines
Google, Yahoo, MSN...

Search Marketing
Marketing, Budget, Planning...

Pay Per Click
Bid, Price, Quality...
SEO Companies
Optimization, Manage, Company...

SEO Tools
Track, Search, Create...

Analytics
Statistics, Counter...
» Submit your site for FREE «

Latest News

Get Your Site Submitted for Free in the World's Largest B2B Directory!

Email Address:
* URL:
*
*Indicates Mandatory Field

Terms & Conditions



Titan Quest Forum Nintendo Wii Graphics Forum
Halo 3 Forum Mac Software

Privacy Policy Legal Sitemap Contact Us RSS Feeds Newsletter Archive SearchNewz.com Privacy Policy Legal Sitemap Contact Us RSS Feeds Newsletter Signup Subscribe to our feeds!