开发者

Are there examples of WWW::Curl::Multi?

开发者 https://www.devze.com 2022-12-12 05:10 出处:网络
Does somebody have a wor开发者_运维问答king example of using WWW::Curl::Multi?Did you try looking on the curl homepage? The CPAN documentation says

Does somebody have a wor开发者_运维问答king example of using WWW::Curl::Multi?


Did you try looking on the curl homepage? The CPAN documentation says

This module provides a Perl interface to libcurl. It is not intended to be a standalone module and because of this, the main libcurl documentation should be consulted for API details at http://curl.haxx.se. The documentation you're reading right now only contains the Perl specific details, some sample code and the differences between the C API and the Perl one.

However, there is an example of both WWW::Curl::Easy and WWW::Curl::Multi on the main CPAN page: http://search.cpan.org/perldoc?WWW::Curl#WWW::Curl::Multi


To give you insight into the community: I've never seen WWW::Curl::Multi in use. The overwhelming majority satisfy their parallel http needs with either POE::Component::Client::HTTP (for ftp and others there are other clients), or Parallel::ForkManager and LWP. Not to say that curl isn't great and that the added multi-network functionality isn't nice, just to say in practice the other two implementations are much more common.

I want to acknowledge that curl multi works great for the php community, and I know nothing bad per se about the perl bindings.


The test suite for a module is often a good place to look for examples.


Take a look at Net::Curl::Multi, I am experimenting with it now. It works at least, where the www version just crashed. I used the latest github version of the libcurl and had to remove the version checks from the Net::Curl Makefile.pl This example does a post to a server for looking up

use Net::Curl::Multi qw(:constants);
use Net::Curl::Easy qw(:constants);
my $start =shift;
my $stop  =shift;
my $usetor=1;
my $outdir = "outtest";
if (! -d $outdir)
{
    mkdir $outdir or die "cannot makedir " . $outdir;
}
my $multi = Net::Curl::Multi->new();

foreach my $id ($start ... $stop)
{
    my $file = $outdir . "/body_$id.html";
    if (
    (!-f $file)     ||
    (-z $file)  
    )
    {

    my $easy = Net::Curl::Easy->new();

    if ($usetor)
    {
        $easy->setopt(CURLOPT_PROXYTYPE, CURLPROXY_SOCKS4); 
        $easy->setopt(CURLOPT_PROXY,"localhost:9050"); #tor
    }


    $easy->setopt(CURLOPT_FOLLOWLOCATION, 1);
    $easy->setopt(CURLOPT_TIMEOUT, 30);
    $easy->setopt(CURLOPT_URL, 'http://www.example.com/searchform');

    $easy->setopt(CURLOPT_CUSTOMREQUEST,"POST" );

    # extract these by looking at the headers
        $easy->setopt(CURLOPT_POSTFIELDS,"ScriptManager=......&__EVENTTARGET=&__EVENTARGUMENT=&__VIEWSTATE=&SearchField=${id}&OtherField=&SomeField=&ScrollTop=&__dnnVariable=&__VIEWSTATEENCRYPTED=&ButtonName=Search");

    $easy->setopt(CURLOPT_WRITEFUNCTION, 
              sub { 
              my $self=shift;
              my $data=shift;
              my $len = length($data);
              open BODY, ">>$file" or die "cannot open $file";            
              print BODY $data;
              close BODY;   
              warn "Write called: for $file and $len";
              return $len;
              }     
        );

    $multi->add_handle( $easy );
    }
}

 my $running = 0;
 do {
     my ($r, $w, $e) = $multi->fdset();
     my $timeout = $multi->timeout();
     select $r, $w, $e, $timeout / 1000
         if $timeout > 0;

     $running = $multi->perform();
     while ( my ( $msg, $easy, $result ) = $multi->info_read() ) {
     warn "finished $msg $easy $result";
         $multi->remove_handle( $easy );

         # process $easy
     }
 } while ( $running );
0

精彩评论

暂无评论...
验证码 换一张
取 消