开发者

UnicodeDecodeError problem with mechanize [duplicate]

开发者 https://www.devze.com 2022-12-21 06:20 出处:网络
This question already has answers here: How to determine the encoding of text (16 answers) Closed 5 years ago.
This question already has answers here: How to determine the encoding of text (16 answers) Closed 5 years ago.

I receive the following string from one website via mechanize:

'We\x92ve'

I know that \x92 stands for character. I'm trying to convert that string to Unicode:

>>开发者_如何学JAVA unicode('We\x92ve','utf-8')
UnicodeDecodeError: 'utf8' codec can't decode byte 0x92 in position 2: unexpected code byte

What am I doing wrong?

Edit: The reason I tried 'utf-8' was this:

>> response = browser.response()
>> response.info()['content-type']
'text/html; charset=utf-8'

Now I see I can't always trust content-type header.


\x92 stands for alright, but it does so in the Windows-1252 encoding, not in UTF-8:

>>> print unicode('We\x92ve','1252')
We’ve

If you don't know what encoding your source data is in, you can detect it using chardet (extremely easy to use).

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号