Recursively list files ordered by file size in HDFS folder

Question

Is there a script for a link if I need to recursively display files ordered by file size in the HDFS folder?

thanks in advance Lin

+3

Lin ma May 22 '15 @ 4:21 am

2 answers

Although @kostya gave a great answer, using a little awk can help reorder the output to improve the downstream process

hdfs dfs -ls -R ${hdfspath} 2> ${hdfsname}.err \
|awk "{ printf \"%s:%s:%s:%s:%s\n\",\$5,\$6,\$3,\$1,\$8; }" \
|sort -rn > hdfs-${hdfsname}.du

0

Chuckcottrill May 25 '17 at 5:38

kostya · Accepted Answer · 2015-05-22T04:32:33+0000

bin/hdfs dfs -ls -R / | sort -r -n -k 5