Dstream apache spark input layout

I am trying to mock the input dstream while writing a unit test spark stream. I can mock the RDDs, but when I try to convert them to dstream, the dstream object becomes empty. I used the following code -

val lines = mutable.Queue[RDD[String]]()
val dstream = streamingContext.queueStream(lines)

// append data to DStream
lines += sparkContext.makeRDD(Seq("To be or not to be.", "That is the question."))

      

Any help regarding the same would be much appreciated.

+3


source to share





All Articles