How to preserve namespace information when parsing HTML with lxml?_问答_开发者

How to preserve namespace information when parsing HTML with lxml?

开发者 https://www.devze.com 2023-03-17 22:19 出处：网络

>>> from lxml.etree import HTML, tostring >>> tostring(HTML(\'<fb:like>\')) \'<html><body><like/></body></html>\'

>>> from lxml.etree import HTML, tostring
>>> tostring(HTML('<fb:like>'))
'<html><body><like/></body></html>'

Note 开发者_StackOverflow社区how the tag turns from <fb:like> to simply <like>.

This makes processing pages that incorporate XFBML with lxml much harder. (Same thing happens to <g:plusone></g:plusone>)

Any help is appreciated.

One way to fix this issue is to patch libxml2.

Referring to the source code of libxml2.9.2 (https: //git.gnome.org/browse/libxml2/tree/?id=v2.9.2), in SAX2.c (https: //git.gnome.org/browse/libxml2/tree/SAX2.c?id=v2.9.2) (the internal SAX parser used to create the DOM tree) at line 1699 attributes with xmlns are not parsed when in HTML mode, and they are parsed like any other attributes at line and 1740. Consequently, it makes sense to adjust line 1622, which splits the name into prefix and local part. Change:

name = xmlSplitQName(ctxt, fullname, &prefix);

into

if (!ctxt->html) {
    name = xmlSplitQName(ctxt, fullname, &prefix);
} else {
    name = xmlStrdup(fullname);
    prefix = NULL;
}

Then libxml2 will consider tags such as <o:p> to be for elements with name o:p, that is, the colon is included in the element name with no special meaning. This is the correct interpretation in HTML. For example, the HTML5 specification says:

In the HTML syntax, namespace prefixes and namespace declarations do not have the same effect as in XML. For instance, the colon has no special meaning in HTML element names.

Hopefully this change will be approved for a future version of libxml2. There is an open bug report (https: //bugzilla.gnome.org/show_bug.cgi?id=654146).

Try adding the namespace prefix definitions that are missing. lxml will avoid the namespaces otherwise, supposedly to make it easier for you.

Most likely the sites you try to parse will not contain these namespace definitions, so you should add them.

Something like this: xmlns:adlcp="http://xxx/yy/zzz"