开发者

Scraping site that requires Javascript enabled with Mechanize + BeautifulSoup (Python)

开发者 https://www.devze.com 2023-01-23 15:36 出处:网络
So.. i got this site I am tryign to scrape, but as I understand lack of support of mechanize for .js, and a stuborn site th开发者_运维知识库at requires javascript enabled browser is

So.. i got this site I am tryign to scrape, but as I understand lack of support of mechanize for .js, and a stuborn site th开发者_运维知识库at requires javascript enabled browser is not a good mix...

I am looking for ideas, on how to do this...

URL : https://members.iracing.com/membersite/login.jsp


Depending on what you need to do, you could use webkit to parse the page, which will allow you to get the final html after the javascript has been executed. You could then use any decent html parser, beautifulsoup for example, to do the rest.


With JavaScript I use Chickenfoot for simple websites and Webkit for more complex.

0

精彩评论

暂无评论...
验证码 换一张
取 消