Use existing column in the date_add () emission function

I want to use an existing light info frame column in a function date_add()

. It works:

spark.range(1, 10).toDF()
.withColumn("Date", current_date())
.withColumn("newdate", expr("date_sub(Date, id)") )
.collect()

      

        
        
        
      

    

but I don't particularly like that, because using it expr()

means I am not getting compile-time checking on date_add and its parameters.

I wanted to refer to the column id

like this:

spark.range(1, 10).toDF()
.withColumn("Date", current_date())
.withColumn("newdate", date_add(col("Date"), col("id")))
.collect()

      

        
        
        
      

    

but that doesn't work, I get a type mismatch error because it date_add()

expects Int

, not org.apache.spark.sql.Column

.

How can I refer to a column id

in my function date_add()

without resorting to expr()

? This comment suggests that I should do this, but cannot figure out how to do it.