Dstream apache spark input layout
I am trying to mock the input dstream while writing a unit test spark stream. I can mock the RDDs, but when I try to convert them to dstream, the dstream object becomes empty. I used the following code -
val lines = mutable.Queue[RDD[String]]()
val dstream = streamingContext.queueStream(lines)
// append data to DStream
lines += sparkContext.makeRDD(Seq("To be or not to be.", "That is the question."))
Any help regarding the same would be much appreciated.
+3
source to share
No one has answered this question yet
Check out similar questions: