Bandwidth issue

I am doing distcp from one haop cluster (version 0.20.2) to another hasoop cluster (version 2.2.0) using below command.

hadoop distcp -update -skipcrccheck "hftp://x.x.x.x:50070//hive/warehouse//staging_eventlog_arpu_comma" "hdfs://y.y.y.y:9000//hive/warehouse/staging_eventlog_arpu_comma"

      

therefore, bandwidth usage must be the source for the destination. but network utilization is greater from destination to source compared to original destination.

hasoop distcp -bandwidth indicates:

**Each map will be restricted to consume only the specified bandwidth. This is not always exact. The map throttles back its bandwidth consumption during a copy, such that the net bandwidth used tends towards the specified value.**

      

So what is it throttling back.?

+3


source to share





All Articles