开发者

how to detect WebPage charset,and get page content?

开发者 https://www.devze.com 2023-01-12 21:09 出处:网络
i开发者_JAVA技巧 use follows code to get page content: URL url=new URL(\"http://www.google.com.hk/intl/zh-CN/privacy.html\");

i开发者_JAVA技巧 use follows code to get page content:

URL url=new URL("http://www.google.com.hk/intl/zh-CN/privacy.html");
BufferedReader reader = new BufferedReader(new InputStreamReader(url.openConnection().getInputStream()));
for(String line=reader.readLine();line!=null;line=reader.readLine()){
    System.out.println(line);
}
reader.close();

page: http://www.google.com.hk/intl/zh-CN/privacy.html charset is "UTF-8",but my system default charset is "GBK",so, these code can't type right.

i know ,i can write a charsetname in InputStreamReader constructor:

new InputStreamReader(url.openConnection().getInputStream(),"UTF-8")

it's will be ok, but i want to know:

how to detect charset,and get page content? (not send two requests better)

any java library can do this? (get webpage content,and don't need set charsetname)

thanks for help :)


There is really no easy way to detect the proper charset. You can hope that the web page you are interested in declares the charset using a <meta charset="utf-8"> tag. When you detect that tag you could switch charset of your parsing.

There are also some libraries that make an effort to detect the charset, for example http://jchardet.sourceforge.net/.

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号