bots and 301 redirect

I have changed the structure of the URLs of my site more than 6 months ago. I detect the use of legacy URLs and redirect to the new URL with a 301 status code. I verified with flidder that the status code is correctly returned upon the request. But bots (yahoo slurps, googlebot, etc.) are still hitting the old URLs. Is there something I am miss开发者_Python百科ing?

---

No, just it takes a very, very long time for crawlers to get the message. I have bots crawling addresses that have not existed since 2005 - when folk harp on with addresses being permanent, they really are.

Additionally, depending on how your URL's are structured, you can disallow the old addresses with robots.txt

---

Try this and this will only redirect to the bots.

if (preg_match("#(google|slurp@inktomi|yahoo! slurp|msnbot)#si", $_SERVER['HTTP_USER_AGENT'])) {
    header("HTTP/1.1 301 Moved Permanently");
    header("Location: http://www.your-main-site.com/");
    exit;
}

---

If external sites have linked to your old pages and those links are still accessible for bots, the bots will keep coming and try to access the content.

---

mentioned you site address here:

http://www.your-main-site.com/

Thats we use to transfer the domain and sometime for blackhat seo.