Exception Null Pointer to toDF ()

Inside the map, I am taking multiple columns and trying to create a DF.

val columns : List[caseclass_Name] = List(CaseClass(id, id)) //Case Class
val splDF = columns.toDF() //DataFrame

      

Mistake:

java.lang.NullPointerException
at org.apache.spark.sql.SQLConf.getConf(SQLConf.scala:638)
at org.apache.spark.sql.SQLConf.dataFrameEagerAnalysis(SQLConf.scala:573)
at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:132)
at org.apache.spark.sql.DataFrame$.apply(DataFrame.scala:52)
at org.apache.spark.sql.SQLContext.createDataFrame(SQLContext.scala:432)
at
org.apache.spark.sql.SQLImplicits.localSeqToDataFrameHolder(SQLImplicits.scala:164)
at package.ObjectName$.Method(Object_name.scala:322)
at)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
at
org.apache.spark.sql.execution.aggregate.TungstenAggregationIterator.processInputs(TungstenAggregationIterator.scala:505)
at
org.apache.spark.sql.execution.aggregate.TungstenAggregationIterator.<init>(TungstenAggregationIterator.scala:686)
at
org.apache.spark.sql.execution.aggregate.TungstenAggregate$$anonfun$doExecute$1$$anonfun$2.apply(TungstenAggregate.scala:95)

      

+3
apache-spark


source to share


No one has answered this question yet

Check out similar questions:

nine
jode DateTime format throws null pointer error in spark RDD functions
2
Can't store java date in Cassandra
2
Exception exception from string stream in thread-client mode
1
How to create Spark DataFrame from RDD [Row] when Row contains Map [Map]
1
Logistic Regression: java.lang.NumberFormatException
0
spark data conversion, JSON to ORC conversion, matches "columnar ambiguous exception"
0
Why is spark throwing ArrayIndexOutOfBoundsException error for empty attributes?
0
How to create Spark DataFrame from RDD [Row] when Row contains complex types
0
java.lang.ClassCastException: org.apache.spark.sql.catalyst.expressions.GenericRow cannot be cast to scala.collection.Iterator
-2
Excluding Null Pointer in the Map function of Spark



All Articles
Loading...
X
Show
Funny
Dev
Pics