I have created a website about some beer brand and had to include age verification page. The verification script is written in PHP and uses sessions to store verification variable. The script works the way that no matter form which link you will try to ente开发者_如何学Pythonr the website it will take you to the verification page first. The verification is very simple. There are 2 button: "I'm under 21" and "I'm over 21". If you click the latter, you can browse the website.
After some time I discovered that the web crawlers are not able to get past verification page. I checked the website in Google webmaster tools and the only text content scanned was from the verification page.
I read somewhere that crawlers are not able to submit form buttons, is it true?
Considering the fact that age verification pages are useless anyways, maybe I should just leave it as a starting page but don't forbid going around it, e.g. from links to the subpages?
Why not make the buttons links instead of submit buttons.
Just have your age verification page detect the major crawler user agents and redirect to a main content page. You can set whatever variables are necessary automatically in the same code block.
精彩评论