Recursively list files ordered by file size in HDFS folder
Is there a script for a link if I need to recursively display files ordered by file size in the HDFS folder?
thanks in advance Lin
+3
Lin ma
source
to share
2 answers
bin/hdfs dfs -ls -R / | sort -r -n -k 5
+4
kostya
source
to share
Although @kostya gave a great answer, using a little awk can help reorder the output to improve the downstream process
hdfs dfs -ls -R ${hdfspath} 2> ${hdfsname}.err \
|awk "{ printf \"%s:%s:%s:%s:%s\n\",\$5,\$6,\$3,\$1,\$8; }" \
|sort -rn > hdfs-${hdfsname}.du
0
Chuckcottrill
source
to share