Start Spark via Mesos
I am trying to start Spark via Mesos cluster, I successfully start light dispatcher via mesos master and register spark structure. But when I try to start any work (spark-feed or spark sheath), initializing stop and doing nothing.
I0708 16:30:43.319941 21373 sched.cpp:157] Version: 0.22.1
I0708 16:30:43.331514 21375 sched.cpp:254] New master detected at master@host
I0708 16:30:43.333658 21375 sched.cpp:264] No credentials provided. Attempting to register without authentication
After that, the client will freeze and that's it.
+3
source to share
1 answer
I solved the same problem with the suggestion suggested by @besil :
Step1: start the dispatcher using:
--master mesos://zk://<master_host>:2181/mesos
Step 2: send apps using
spark-submit --deploy-mode cluster <other options> --master mesos://<dispatcher_ip>:7077 --class <class_name> <jar>
Hope it helps
+2
source to share