How do I use the BLAS library in Spark?
I am new to scala and I am writing Spark application in scala and I need to use a function axpy
from org.apache.spark.mllib.linalg.BLAS
. However, it looks inaccessible to users. Instead, I am trying to import com.github.fomil.netlib
and access it directly. But I could too. I need to multiply by DenseVector.
source to share
Currently, the BLAS class inside mllib is tagged private[spark]
in the spark source code . This means that it is not accessible from the outside to spark, as you seem to understand. In short, you cannot use it in your code.
If you want to use netlib-java classes directly, you need to add the following dependency to your project
libraryDependencies += "com.github.fommil.netlib" % "all" % "1.1.2" pomOnly()
This should allow you to import the BLAS class. Note, I haven't really tried to use it, but I can accomplish BLAS.getInstance()
with no problem. There may be some complications when installing on some Linux platforms, as described here - https://github.com/fommil/netlib-java .
source to share