Grab all text from html with Html Agility Pack_问答_开发者

Grab all text from html with Html Agility Pack

开发者 https://www.devze.com 2023-01-25 07:03 出处：网络

Input <html><body><p>foo <a href=\'http://www.example.com\'>bar</a> baz</p></body></html>

Input

<html><body><p>foo <a href='http://www.example.com'>bar</a> baz</p></body></html>

Output

foo
bar
baz

I know of htmldoc.Docume开发者_如何学运维ntNode.InnerText, but it will give foobarbaz - I want to get each text, not all at a time.

XPATH is your friend :)

HtmlDocument doc = new HtmlDocument();
doc.LoadHtml(@"<html><body><p>foo <a href='http://www.example.com'>bar</a> baz</p></body></html>");

foreach(HtmlNode node in doc.DocumentNode.SelectNodes("//text()"))
{
    Console.WriteLine("text=" + node.InnerText);
}

var root = doc.DocumentNode;
var sb = new StringBuilder();
foreach (var node in root.DescendantNodesAndSelf())
{
    if (!node.HasChildNodes)
    {
        string text = node.InnerText;
        if (!string.IsNullOrEmpty(text))
            sb.AppendLine(text.Trim());
    }
}

This does what you need, but I am not sure if this is the best way. Maybe you should iterate through something other than DescendantNodesAndSelf for optimal performance.

I was in the need of a solution that extracts all text but discards the content of script and style tags. I could not find it anywhere, but I came up with the following which suits my own needs:

StringBuilder sb = new StringBuilder();
IEnumerable<HtmlNode> nodes = doc.DocumentNode.Descendants().Where( n => 
    n.NodeType == HtmlNodeType.Text &&
    n.ParentNode.Name != "script" &&
    n.ParentNode.Name != "style");
foreach (HtmlNode node in nodes) {
    Console.WriteLine(node.InnerText);

var pageContent = "{html content goes here}";
var pageDoc = new HtmlDocument();
pageDoc.LoadHtml(pageContent);
var pageText = pageDoc.DocumentNode.InnerText;

The specified example for html content:

<html><body><p>foo <a href='http://www.example.com'>bar</a> baz</p></body></html>

will produce the following output:

foo bar baz

public string html2text(string html) {
    HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();
    doc.LoadHtml(@"<html><body>" + html + "</body></html>");
    return doc.DocumentNode.SelectSingleNode("//body").InnerText;
}

This workaround is based on Html Agility Pack. You can also install it via NuGet (package name: HtmlAgilityPack).

https://github.com/jamietre/CsQuery

have you tried CsQuery? Though not being maintained actively - it's still my favorite for parsing HTML to Text. Here's a one liner of how simple it is to get the Text from HTML.

var text = CQ.CreateDocument(htmlText).Text();

Here's a complete console application:

using System;
using CsQuery;

public class Program
{
    public static void Main()
    {
        var html = "<div><h1>Hello World <p> some text inside h1 tag under p tag </p> </h1></div>";
        var text = CQ.CreateDocument(html).Text();
        Console.WriteLine(text); // Output: Hello World  some text inside h1 tag under p tag

    }
}

I understand that OP has asked for HtmlAgilityPack only but CsQuery is another unpopular and one of the best solutions I've found and wanted to share if someone finds this helpful. Cheers!

I just changed and fixed some people's answers to work better:

var document = new HtmlDocument();
        document.LoadHtml(result);
        var sb = new StringBuilder();
        foreach (var node in document.DocumentNode.DescendantsAndSelf())
        {
            if (!node.HasChildNodes && node.Name == "#text" && node.ParentNode.Name != "script" && node.ParentNode.Name != "style")
            {
                string text = node.InnerText?.Trim();
                if (text.HasValue() && !text.StartsWith('<') && !text.EndsWith('>'))
                    sb.AppendLine(System.Web.HttpUtility.HtmlDecode(text.Trim()));
            }
        }