How do I build and debug a spark from an IDE (Eclipse preferred)?

I wanted to contribute to the spark.

I have cloned the git repository locally. Please suggest setting up the spark first and then launching the welcome world from the IDE itself.

+3


source to share


2 answers


Follow this tutorial to import / create Spark in IntelliJ or Eclipse.

If you're interested in helping Spark visit this wiki page for more information:



+1


source


I assume you already have the latest version of the Scala IDE (4.0 for now) from scala-ide.org .

  • exporting projects using sbt eclipse

    , I think you figured that out already.
  • import all projects to workspace ( Import Existing projects

    )
    • you will probably see a number of errors related to "cross compiled libraries"

If you want to upgrade to Scala 2.10, you need to tweak your Scala installation to the exact version of Scala used to compile Spark. At the time of this writing, Scala 2.10.4.



  • you can do this by Eclipse Preferences -> Scala -> Installations

    pointing to the directory of lib/

    your Scala 2.10.4 distribution.
  • select all Spark projects and right click select Scala -> Set Scala Installation

    and point to install 2.10.4. This should remove any errors regarding invalid cross-compiled libraries.
  • a clean build should be successful.

You can easily find examples when running Spark, for example here . You can launch the Spark application with right-click -> Run As Scala Application

.

+1


source







All Articles