开发者

How can I tell how a website is determining if the browser has JS enabled?

开发者 https://www.devze.com 2023-03-05 13:58 出处:网络
We have a Perl script that emulates a browser using LWP::UserAgent to login to a website and download some data. Recently that website was changed so that you have to have JavaScript enabled in order

We have a Perl script that emulates a browser using LWP::UserAgent to login to a website and download some data. Recently that website was changed so that you have to have JavaScript enabled in order to log i开发者_如何学JAVAn. How can we tell what the website is doing to determine whether JavaScript is enabled, and/or how can we use LWP::UserAgent to spoof the site into thinking it is a JavaScript enabled browser?


Most websites only require javascript to be enabled to log in if it's actually needed for the log in to succeed. They might use javascript to set a cookie or display the log in form. Maybe the log in is done using an XMLHTTPRequest. In most of these cases, you won't be able to spoof it; you'll actually need a javascript-enabled browser. Get an embed-able version of webkit or something.


It turns out that the problem we were seeing was not with the Perl script at all...it was an error in a configuration file.

While researching the problem, someone suggested a great tool called wsp Web Scraping Proxy. That was a great suggestion to find out exactly what was going on between the server and the browser.

Thanks for taking the time to answer the question.

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号