Exception Null Pointer to toDF ()

Inside the map, I am taking multiple columns and trying to create a DF.

val columns : List[caseclass_Name] = List(CaseClass(id, id)) //Case Class
val splDF = columns.toDF() //DataFrame

      

Mistake:

java.lang.NullPointerException
at org.apache.spark.sql.SQLConf.getConf(SQLConf.scala:638)
at org.apache.spark.sql.SQLConf.dataFrameEagerAnalysis(SQLConf.scala:573)
at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:132)
at org.apache.spark.sql.DataFrame$.apply(DataFrame.scala:52)
at org.apache.spark.sql.SQLContext.createDataFrame(SQLContext.scala:432)
at
org.apache.spark.sql.SQLImplicits.localSeqToDataFrameHolder(SQLImplicits.scala:164)
at package.ObjectName$.Method(Object_name.scala:322)
at)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
at
org.apache.spark.sql.execution.aggregate.TungstenAggregationIterator.processInputs(TungstenAggregationIterator.scala:505)
at
org.apache.spark.sql.execution.aggregate.TungstenAggregationIterator.<init>(TungstenAggregationIterator.scala:686)
at
org.apache.spark.sql.execution.aggregate.TungstenAggregate$$anonfun$doExecute$1$$anonfun$2.apply(TungstenAggregate.scala:95)

      

+3


source to share





All Articles