开发者

How I can get web page's content and save it into the string variable

开发者 https://www.devze.com 2023-01-31 15:24 出处:网络
How I can g开发者_JAVA百科et the content of the web page using ASP.NET?I need to write a program to get the HTML of a webpage and store it into a string variable.You can use the WebClient

How I can g开发者_JAVA百科et the content of the web page using ASP.NET? I need to write a program to get the HTML of a webpage and store it into a string variable.


You can use the WebClient

Using System.Net;

using(WebClient client = new WebClient()) {
    string downloadString = client.DownloadString("http://www.gooogle.com");
}


I've run into issues with Webclient.Downloadstring before. If you do, you can try this:

WebRequest request = WebRequest.Create("http://www.google.com");
WebResponse response = request.GetResponse();
Stream data = response.GetResponseStream();
string html = String.Empty;
using (StreamReader sr = new StreamReader(data))
{
    html = sr.ReadToEnd();
}


I recommend not using WebClient.DownloadString. This is because (at least in .NET 3.5) DownloadString is not smart enough to use/remove the BOM, should it be present. This can result in the BOM () incorrectly appearing as part of the string when UTF-8 data is returned (at least without a charset) - ick!

Instead, this slight variation will work correctly with BOMs:

string ReadTextFromUrl(string url) {
    // WebClient is still convenient
    // Assume UTF8, but detect BOM - could also honor response charset I suppose
    using (var client = new WebClient())
    using (var stream = client.OpenRead(url))
    using (var textReader = new StreamReader(stream, Encoding.UTF8, true)) {
        return textReader.ReadToEnd();
    }
}


Webclient client = new Webclient();
string content = client.DownloadString(url);

Pass the URL of page who you want to get. You can parse the result using htmlagilitypack.


I have always been using WebClient, but at the time this post is made (.NET 6 is avail), WebClient is getting deprecated.

The preferred way is

HttpClient client = new HttpClient();
string content = await client.GetStringAsync(url);
0

精彩评论

暂无评论...
验证码 换一张
取 消