BeautifulSoup Error (CGI Escape)_问答_开发者_运维开发者技术经验分享

开发者 https://www.devze.com 2023-03-02 01:09 出处：网络

Getting the following error: Traceback (most recent call last): File \"stack.py\", line 31, in ? print >> out, \"%s\" % escape(p)File

相关专题：python

Getting the following error:

Traceback (most recent call last):
File "stack.py", line 31, in ?
print >> out, "%s" % escape(p) File
"/usr/lib/python2.4/cgi.py", line
1039, in escape
s = s.replace("&", "&") # Must be done first! TypeError: 'NoneType'
object is not callable

For the following code:

import urllib2
from cgi import escape  # Important!
fro开发者_开发百科m BeautifulSoup import BeautifulSoup

def is_talk_anchor(tag):
return tag.name == "a" and tag.findParent("dt", "thumbnail")

def talk_description(tag):
return tag.name == "p" and tag.findParent("h3")

links = []
desc = []

for pagenum in xrange(1, 5):
soup = BeautifulSoup(urllib2.urlopen("http://www.ted.com/talks?page=%d" % pagenum))
links.extend(soup.findAll(is_talk_anchor))
page = BeautifulSoup(urllib2.urlopen("http://www.ted.com/talks/arvind_gupta_turning_trash_into_toys_for_learning.html"))
desc.extend(soup.findAll(talk_description))

out = open("test.html", "w")

print >>out, """<html><head><title>TED Talks Index</title></head>
<body>
<table>
<tr><th>#</th><th>Name</th><th>URL</th><th>Description</th></tr>"""

for x, a in enumerate(links):
  print >> out, "<tr><td>%d</td><td>%s</td><td>http://www.ted.com%s</td>" % (x + 1, escape(a["title"]), escape(a["href"]))

for y, p in enumerate(page):
  print >> out, "<td>%s</td>" % escape(p)

print >>out, "</tr></table>"

I think the issue is with % escape(p). I'm trying to take the contents of that <p> out. Am I not supposed to use escape?

Also having an issue with the line:

page = BeautifulSoup(urllib2.urlopen("%s") % a["href"])

That's what I want to do, but again running into errors and wondering if there's an alternate way of doing it. Just trying to collect the links I found from previous lines and run it through BeautifulSoup again.

You have to investigate (using pdb) why one of your links is returned as None instance.

In particular: the traceback is self-speaking. The escape() is called with None. So you have to investigate which argument is None...it's one of of your items in 'links'. So why is one of your items None?

Likely because one of your calls to

def is_talk_anchor(tag):
   return tag.name == "a" and tag.findParent("dt", "thumbnail")

returns None because tag.findParent("dt", "thumbnail") returns None (due to your given HTML input).

So you have to check or filter your items in 'links' for None (or adjust your parser code above) in order to pickup only existing links according to your needs.

And please read your tracebacks carefully and think about what the problem might be - tracebacks are very helpful and provide you with valuable information about your problem.