Does somebody have a wor开发者_运维问答king example of using WWW::Curl::Multi?
Did you try looking on the curl homepage? The CPAN documentation says
This module provides a Perl interface to libcurl. It is not intended to be a standalone module and because of this, the main libcurl documentation should be consulted for API details at http://curl.haxx.se. The documentation you're reading right now only contains the Perl specific details, some sample code and the differences between the C API and the Perl one.
However, there is an example of both WWW::Curl::Easy
and WWW::Curl::Multi
on the main CPAN page: http://search.cpan.org/perldoc?WWW::Curl#WWW::Curl::Multi
To give you insight into the community: I've never seen WWW::Curl::Multi
in use. The overwhelming majority satisfy their parallel http needs with either POE::Component::Client::HTTP
(for ftp and others there are other clients), or Parallel::ForkManager
and LWP
. Not to say that curl isn't great and that the added multi-network functionality isn't nice, just to say in practice the other two implementations are much more common.
I want to acknowledge that curl multi works great for the php community, and I know nothing bad per se about the perl bindings.
The test suite for a module is often a good place to look for examples.
Take a look at Net::Curl::Multi, I am experimenting with it now. It works at least, where the www version just crashed. I used the latest github version of the libcurl and had to remove the version checks from the Net::Curl Makefile.pl This example does a post to a server for looking up
use Net::Curl::Multi qw(:constants);
use Net::Curl::Easy qw(:constants);
my $start =shift;
my $stop =shift;
my $usetor=1;
my $outdir = "outtest";
if (! -d $outdir)
{
mkdir $outdir or die "cannot makedir " . $outdir;
}
my $multi = Net::Curl::Multi->new();
foreach my $id ($start ... $stop)
{
my $file = $outdir . "/body_$id.html";
if (
(!-f $file) ||
(-z $file)
)
{
my $easy = Net::Curl::Easy->new();
if ($usetor)
{
$easy->setopt(CURLOPT_PROXYTYPE, CURLPROXY_SOCKS4);
$easy->setopt(CURLOPT_PROXY,"localhost:9050"); #tor
}
$easy->setopt(CURLOPT_FOLLOWLOCATION, 1);
$easy->setopt(CURLOPT_TIMEOUT, 30);
$easy->setopt(CURLOPT_URL, 'http://www.example.com/searchform');
$easy->setopt(CURLOPT_CUSTOMREQUEST,"POST" );
# extract these by looking at the headers
$easy->setopt(CURLOPT_POSTFIELDS,"ScriptManager=......&__EVENTTARGET=&__EVENTARGUMENT=&__VIEWSTATE=&SearchField=${id}&OtherField=&SomeField=&ScrollTop=&__dnnVariable=&__VIEWSTATEENCRYPTED=&ButtonName=Search");
$easy->setopt(CURLOPT_WRITEFUNCTION,
sub {
my $self=shift;
my $data=shift;
my $len = length($data);
open BODY, ">>$file" or die "cannot open $file";
print BODY $data;
close BODY;
warn "Write called: for $file and $len";
return $len;
}
);
$multi->add_handle( $easy );
}
}
my $running = 0;
do {
my ($r, $w, $e) = $multi->fdset();
my $timeout = $multi->timeout();
select $r, $w, $e, $timeout / 1000
if $timeout > 0;
$running = $multi->perform();
while ( my ( $msg, $easy, $result ) = $multi->info_read() ) {
warn "finished $msg $easy $result";
$multi->remove_handle( $easy );
# process $easy
}
} while ( $running );
精彩评论