开发者

What is called first - robots.txt or mod_rewrite in htaccess

开发者 https://www.devze.com 2023-02-17 05:08 出处:网络
I need some help. I\'m not sure about the order on request for mod开发者_运维问答_rewrite and robots.txt.

I need some help. I'm not sure about the order on request for mod开发者_运维问答_rewrite and robots.txt.

Some urls belong to a rewrite rule:

/index.php?id=123 to /home

Other urls don't have a rewrite:

/index.php?id=444

I made this entry to my robots.txt:

User-agent: *
Disallow: /index.php?id

Will the site with /home be indexed by search engines?


The robots.txt file is interpreted by the client (spider), and they don't know what rewrites you have in your system. Thus, spiders would not fetch URLs from your site if they look like the pattern in robots.txt but would if they found the same content through /home.

0

精彩评论

暂无评论...
验证码 换一张
取 消