开发者

Jsoup (connect) Java takes a long time

开发者 https://www.devze.com 2023-03-20 20:21 出处:网络
I\'m trying to get a few lines of code from many html pages. I\'m using Jsoup and it takes over 2 minutes to go over 70+ html pages (90% of the time is spent on creating the new Document through jsou

I'm trying to get a few lines of code from many html pages.

I'm using Jsoup and it takes over 2 minutes to go over 70+ html pages (90% of the time is spent on creating the new Document through jsoup.connect).

Is this normal? Would using another parser make any difference?

Document doc = Jsoup.connect(Url).timeout开发者_运维百科(10*1000).get();

I'm able to go through all the webpages.


The jsoup normally takes about 2 seconds to make connetion and to fetch html content

In case if the url is redirected to another site,then it may take up to 5 seconds.

May be some of the pages which you fetch may be redirected,set the timeout to 3 seconds ,you may know what went wrong ..

0

精彩评论

暂无评论...
验证码 换一张
取 消