开发者

web scraping/parsing of college course site

开发者 https://www.devze.com 2022-12-14 17:39 出处:网络
Trying to parse/scrape the course site for memphis. The site is \"https://spectrumssb2.memphis.edu/pls/PROD/bwckgens.p_proc_term_date\"开发者_StackOverflow. It appears to be some sort of javascript is

Trying to parse/scrape the course site for memphis. The site is "https://spectrumssb2.memphis.edu/pls/PROD/bwckgens.p_proc_term_date"开发者_StackOverflow. It appears to be some sort of javascript issue, or dynamic generation of the text. I can see the underlying DOM structure using livehttpdheaders/Firefox, but not when I simply view the underlying source/text of the page..

Thoughts/Comments/Pointers would be appreciated...


Well this modern days the site may be assembled in few steps. First the main structure is pulled in and then, often based on identity of the user additional AJAX calls are executed. Your best bet is to sniff HTTP to see what kind of requests are issued between the site is initially requested and when it's fully built

Since you are using firebug you can get HttpFox add-on which gives you what you need

0

精彩评论

暂无评论...
验证码 换一张
取 消