Why are we setting up mapred.job.tracker in YARN?
I know YARN is introduced and it has replaced JobTracker and TaskTracker.
I have seen several tutorials on installing Hadoop 2.6.0 / 2.7.0 and they configure mapreduce.framework.name
as yarn property and mapred.job.tracker
as local or host: port.
Property Description mapred.job.tracker
"Host and port on which Tracker Job MapReduce is running. If" local ", then jobs are started in the process as one map and reduce the task."
My doubt is why they are configuring it if we are using YARN, I mean JobTracker shouldn't work correctly?
Forgive me if my question is dumb.
Edit: These are the tutorials I was talking about.
http://chaalpritam.blogspot.in/2015/01/hadoop-260-multi-node-cluster-setup-on.html
http://pingax.com/install-apache-hadoop-ubuntu-cluster-setup/
https://chawlasumit.wordpress.com/2015/03/09/install-a-multi-node-hadoop-cluster-on-ubuntu-14-04/
source to share
This is just a guess, but either the tutorials that talk about setting up JobTracker in YARN are written by people who don't know what YARN is, or they install it if you decide to stop working with YARN someday. You are correct: JobTracker and TaskTracker do not exist in YARN. You can add properties if you like, but they will be ignored. New properties for each of the JobTracker and TaskTracker replacements have been added using YARN, for example yarn.resourcemanager.address
as replacements mapred.jobtracker.address
.
If you list your Java processes when running Hadoop under YARN, you don't see JobTracker
either TaskTracker
:
10561 Jps
20605 NameNode
17176 DataNode
18521 ResourceManager
19625 NodeManager
18424 JobHistoryServer
You can read more about how YARN works here .
source to share