Nodejs + mikeal / Request module, how to close request or increase MaxSockets

I have a Nodejs application that is designed for simple and straightforward end-to-end testing of a large web application. This app uses mikeal / Request and Cheerio to move, request, move and check web pages in the app.

We are refactoring some tests and we are faced with an issue where several functions request

are called sequentially. I believe it might be due to the Node.js process hitting the MaxSockets limit, but I'm not entirely sure.

Some codes ...

var request = require('request');
var cheerio = require('cheerio);
var async = require('async');

var getPages_FromMenuLinks = function() {
    var pageUrl = 'http://www.example.com/app';
    async.waterfall([
        function topPageRequest(cb1) {
            var menuLinks = [];
            request(pageUrl, function(err, resp, page) {
                var $ = cheerio.load(page);
                $('div[class*="sub-menu"]').each(function (i, elem) {
                    menuLinks.push($(this).find('a').attr('href');
                });
                cb1(null, menuLinks);
            });
        }, function subMenuRequests(menuLinks, cb2) {
            async.eachSeries(menuLinks, functionv(link, callback) {
                request(link, function(err, resp, page) {
                    var $ = cheerio.load(page);
                    // do some quick validation testing of elements on the expected page
                    callback();
                });
            }, function() { cb2(null) } );
        }
    ], function () { });
};
module.export = getPages_FromMenuLinks;

      

        
        
        
      

    

Now if I run this Node script it runs through the first one topPageRequest

and runs subMenuRequests

, but then hangs after the request completes for the third submenu item.

It seems that I can use the Max-Sockets constraint, either on Node or on my machine (?) - I am testing this on a standard Windows 8 machine running Node v0.10.26.

I tried using request({pool:{maxSockets:25}, url:link}, function(err, resp...

it but it doesn't seem to make any difference.

It also seems like there is a way to abort the request object if I create it first (as found here ). But I have no idea how to "parse" page

, similar to what happens in the above code. In other words, from the solution found in the link ...

var theRequest = request({ ... });
theRequest.pipe(parser);
theRequest.abort();

      

        
        
        
      

    

... how would I rewrite my code to pipe

and "parse" the request?