Make an HTTP request inside a web worker
I am trying to use web workers or threads in my node app for the first time. I am using webworker-threads npm module .
Basically, I would like each worker to make requests to the server, measure the response time, and send it back to the main thread.
I tried it in different ways, but I just can't get it to work. Basic examples of working with documents. But when I try to request a module ("request" in my case), the workers just stop working, no error messages. I've seen in the docs that it doesn't work inside a worker, so I tried "importScripts ()" which doesn't work either. When using threadpools I tried to use .all.eval () but it didn't work either.
Since this is the first time we are working with web workers / threads in node, I may be misunderstanding how to use these things in general. Here's one example I've tried:
server.js
var Worker = require('webworker-threads').Worker;
var worker = new Worker('worker.js');
worker.js
console.log("before import");
importScripts('./node_modules/request/request.js');
console.log("after import");
This basic example only prints before import
and then stops.
source to share
Website workers are inline javascript, so you can't achieve what you want with them. Worker nicknames do not support node.js api or npm packages (like http or request.js). For concurrency, you don't need multi-threaded magic, just use async.js or promises. If you want to play with threads then child_processes is the way to go. You can also use API to manage child_processes like https://github.com/rvagg/node-worker-farm
Given your example, you could write something like this:
main.js
var workerFarm = require('worker-farm')
, workers = workerFarm(require.resolve('./child'))
, ret = 0;
var urls = ['https://www.google.com', 'http://stackoverflow.com/', 'https://github.com/'];
urls.forEach(function (url) {
workers(url, function (err, res, body, responseTime) {
console.log('Url ' + url + 'finished in ' + responseTime + 'ms');
//Ugly code here use async/promise instead
if (++ret == urls.length)
workerFarm.end(workers);
});
});
child.js
var request = require('request');
module.exports = function(url, cb) {
var start = new Date();
request(url, function(err, res, body) {
var responseTime = new Date() - start;
cb(err, res, body, responseTime);
});
};
source to share