Using Rcpp function in parLapply on Windows

I am doing R code optimization with Rcpp and parallel computing on Windows. I have a problem calling Rcpp function in parLapply. Example following

Rcpp code (test.cpp)

#include <Rcpp.h>
using namespace Rcpp;

// [[Rcpp::export]]
NumericVector payoff( double strike, NumericVector data) {
    return pmax(data - strike, 0);
}

      

R-code

library(parallel)
library(Rcpp)

sourceCpp("test.cpp")

strike_list <- as.list(seq(10, 100, by = 5))

data <- runif(10000) * 50

# One core version
strike_payoff <- lapply(strike_list, payoff, data)

# Multiple cores version
numWorkers <- detectCores()
cl <- makeCluster(numWorkers, type = "PSOCK")
clusterExport(cl = cl,varlist = "payoff")
strike_payoff <- parLapply(cl, strike_list, payoff, data)

      

Error for parallel version

Error in checkForRemoteErrors(val) : 
  8 nodes produced errors; first error: NULL value passed as symbol address   

      

I know this is a Windows problem as mclapply works well on Linux, but I don't have a Linux machine as powerful as Windows.

Any ideas how to fix this?

+3


source to share


1 answer


You need to run a call sourceCpp()

in each spawned process or get some code. Right now, the main process has a function, spawned workers do not.



The easiest way is to create a package and load it with each workflow.

+7


source







All Articles