Scala - Expanding an argument list in a pattern matching expression
I am very new to Scala and am trying to use it as a frontend for Spark. I am facing the problem of generating a generic CSV for the DataFrame function. For example, I have a CSV with about 50 fields, the first being task
, name
and id
. I can get the following to work:
val reader = new CSVReader(new StringReader(txt))
reader.readAll().map(_ match {
case Array(task, name, id, _*) => Row(task, name, id)
case unexpectedArrayForm =>
throw new RuntimeException("Record did not have correct number of fields: "+ unexpectedArrayForm.mkString(","))
})
However, I wouldn't have to hard-code the number of fields needed to create a spark series. I've tried this:
val reader = new CSVReader(new StringReader(txt))
reader.readAll().map(_ match {
case Array(args @ _*) => Row(args)
case unexpectedArrayForm =>
throw new RuntimeException("Record did not have correct number of fields: "+ unexpectedArrayForm.mkString(","))
})
But it just creates a Row object with one item. How can I get it to expand args
into Row(args)
so that if I have an array of N elements, I get a string with N elements?
source to share
This should do the trick:
val reader = new CSVReader(new StringReader(txt))
reader.readAll().map(_ match {
case a: Array[String] => Row(a:_*)
case unexpectedArrayForm =>
throw new RuntimeException("Record did not have correct number of fields: "+ unexpectedArrayForm.mkString(","))
})
Edited to fix missing array type
source to share
Change your input to variable length by adding _*
:
Row(args:_*)
This is what it Row
takes for its signatureapply
.
In fact, you don't even need to do anything other than pass this to a string, since this is already the correct sequence type.
reader.readAll().map(Row(_:_*))
source to share