开发者

WebClient.DownloadString() Not Producing Exact HTML

开发者 https://www.devze.com 2022-12-31 06:13 出处:网络
So here\'s the deal. I\'m creating a spider bot for a website that scans all the product pages and records the product data. I\'m using C# and the WebClient library to download the HTML string. The si

So here's the deal. I'm creating a spider bot for a website that scans all the product pages and records the product data. I'm using C# and the WebClient library to download the HTML string. The site I'm crawling must be specially made because the HTML that is received from开发者_运维百科 WebClient.DownloadString() is different than the HTML that I get when I view the source of the HTML when visiting it on a browser. This seems intentional because the only info I can't get is the price.

Does anyone know a workaround for this problem or can anyone explain what is happening? Thanks.


It is probably using the the user agent string to decide what content to send. The example here shows how to set the user agent header.

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号