开发者

How to download a html page via asp.net

开发者 https://www.devze.com 2023-02-02 11:44 出处:网络
how can i download a web page from my web app, then reading \"title\" and \"开发者_JAVA百科description\" metatag ?

how can i download a web page from my web app, then reading "title" and "开发者_JAVA百科description" metatag ? Like a web crawler, but in Asp.net and called by ..an asp.net web page ?

Thanks!


You can do a screen scrape of an external URL in .NET using the WebClient class, which you'll find in the System.Net namespace. Use the DownloadData method to download the content from a specified URL. The downloaded data comes down as a byte array, but you can convert this to a string.

The following snippet shows how to use WebClient to grab the HTML from my blog's homepage, http://scottonwriting.net/sowblog/default.aspx:

// Create a new WebClient instance.
WebClient myWebClient = new WebClient();

// Download the markup from 
byte[] myDataBuffer = myWebClient.DownloadData("http://scottonwriting.net/sowblog/default.aspx");

// Convert the downloaded data into a string
string markup = Encoding.ASCII.GetString(myDataBuffer);

Once you have the markup you can use regular expressions or string searching methods to pick out the markup of interest.


Use the HTML Agility Pack and its HTMLWeb class.

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号