How do I access the static resources in the jar (which correspond to the src / main / resources folder)?

I have a Spark Streaming application built with Maven (like a jar) and deployed with a spark-submit

script. The app project layout follows the standard catalog layouts:



In an object DoSomething.scala

, I have a method (let it call it doSomething()

) that tries to execute a Perl script -

(from a folder resources

) - using scala.sys.process.Process

and passing two script arguments (the first is the absolute path to the binary used as input, the second is the path / name generated output file). I call then DoSomething.doSomething()


The problem is I was unable to access the script, not absolute paths, relative paths, getClass.getClassLoader.getResource, getClass.getResource, I specified the resources folder in mine pom.xml

. None of my attempts have succeeded. I don't know how to find the stuff I have put in src / main / resources.

Any help would be appreciated.


  • I am using an external Process instead of Spark because at this point in my workflow, I have to process binaries as input and output.
  • I am using Spark-streaming 1.1.0, Scala 2.10.4 and Java 7. I am creating a jar with "Maven" installation from Eclipse (Kepler)
  • When I use the getClass.getClassLoader.getResource

    "standard" method to access resources, I find that the actual classpath is a spark-submit


source to share

1 answer

There are several solutions. The simplest is to use the Scala process framework:

import scala.sys.process._

object RunScript {
  val arg = "some argument"
  val stream = RunScript.getClass.getClassLoader.getResourceAsStream("")
  val ret: Int = (s"/usr/bin/perl - $arg" #< stream).!


In this case ret

- the return code for the process, and any process concluded - stdout


The second (longer) solution is to copy the file

from the jar file to some temporary location and execute it from there. This piece of code should have most of what you need.

object RunScript {
  // Set up copy destination from the Java temporary directory. This is /tmp on Linux
  val destDir = System.getProperty("") + "/"
  // Get a stream to the script in the resources dir
  val source = Channels.newChannel(RunScript.getClass.getClassLoader.getResourceAsStream(""))
  val fileOut = new File(destDir, "")
  val dest = new FileOutputStream(fileOut)
  // Copy file to temporary directory
  dest.getChannel.transferFrom(source, 0, Long.MaxValue)
// Schedule the file for deletion for when the JVM quits
sys.addShutdownHook {
  new File(destDir, "").delete
// Now you can execute the script.


This approach allows you to link your own libraries in JAR files. Copying them allows the libraries to be loaded at runtime for whatever JNI hint you have planned.



All Articles