How do I build and debug a spark from an IDE (Eclipse preferred)?
2 answers
I assume you already have the latest version of the Scala IDE (4.0 for now) from scala-ide.org .
- exporting projects using
sbt eclipse
, I think you figured that out already. - import all projects to workspace (
Import Existing projects
)- you will probably see a number of errors related to "cross compiled libraries"
If you want to upgrade to Scala 2.10, you need to tweak your Scala installation to the exact version of Scala used to compile Spark. At the time of this writing, Scala 2.10.4.
- you can do this by
Eclipse Preferences -> Scala -> Installations
pointing to the directory oflib/
your Scala 2.10.4 distribution. - select all Spark projects and right click select
Scala -> Set Scala Installation
and point to install 2.10.4. This should remove any errors regarding invalid cross-compiled libraries. - a clean build should be successful.
You can easily find examples when running Spark, for example here . You can launch the Spark application with right-click -> Run As Scala Application
.
+1
source to share