CURL multi handler reuse

Well I am trying to reuse the handles I spawned in the first process, however after the first run it just stops working. If I remove (or recreate the entire handler) the handles and add them again, it works great. What could be the culprit for this?

Currently my code looks like this:

<?php
echo 'Handler amount: ';
$threads = (int) trim(fgets(STDIN));
if($threads < 1) {
    $threads = 1;
}

$s = microtime(true);
$url = 'http://mywebsite.com/some-script.php';

$mh = curl_multi_init();
$ch = array();
for($i = 0; $i < $threads; $i++) {
    $ch[$i] = curl_init($url);
    curl_setopt_array($ch[$i], array(
        CURLOPT_USERAGENT => 'Mozilla/5.0 (X11; Linux i686; rv:21.0) Gecko/20130213 Firefox/21.0',
        CURLOPT_REFERER => $url,
        CURLOPT_RETURNTRANSFER => true,
        CURLOPT_NOBODY => true
    ));

    curl_multi_add_handle($mh, $ch[$i]);
}

while($mh) {
    $running = null;
    do {
        curl_multi_exec($mh, $running);
    } while($running > 0);

    $e = microtime(true);
    $totalTime = number_format($e - $s, 2);
    if($totalTime >= 1) {
        echo floor($threads / $totalTime) . ' requests per second (total time '.$totalTime.'s)' . "\r";
        $s = microtime(true);
    }
}

foreach($ch as $handler) {
    curl_multi_remove_handle($mh, $handler);
    curl_close($handler);
}

curl_multi_close($mh);
?>

      

When I have CURLOPT_VERBOSE

it set to true

, I see a lot of messages "extra stuff odd transfer.c: 1037: 0 0", I read about them on another question and it seems like it is caused by some obvious things:

  • Too fast

  • Firewall

  • ISP limitation

AFAIK, this is not the case, because if I recreate the descriptors every time, they successfully complete at about 79 requests per second (about 529 bytes each)

My process for reusing descriptors:

  • Create a multiprocessor and add the specified number of descriptors to the multiprocessor

  • While the mutli handler is running, execute all descriptors

  • After the while loop has stopped (it seems very unlikely that it will), close all handles and the handler with multiple curls

It executes all descriptors once and then stops.

It really shocks me. Any ideas?

+3


source to share





All Articles