Scala Spark RDD current number of sections

Question

How do you determine the number of sections of an arbitrary RDD in Scala?

I know that PySpark RDD has getNumPartitions defined in its API, but I can't find an equivalent on the Scala side.

+3

scala apache-spark

x89a10 04 Jul 15 at 19:11

source to share

2 answers

At least in Spark 1.6.1 this works

rdd.getNumPartitions()

+4

echo May 23 '16 at 7:19

source to share

Gábor Bakos · Accepted Answer · 2015-07-04T19:20:46+0000

It should be rdd.partitions.length

.