Softening of Ahol's tendons
I just gave this Hadoop tuorial read which states that Hadoop has Achilles tendon (single point of failure) inJobTracker
JobTracker is the only point of failure for the Hadoop MapReduce service, which means that if the JobTracker goes down, all jobs in progress are stopped.
And also this article which says what NameNode
is SPOF:
The only point of failure in a Hadoop cluster is the NameNode.
Single points of failure are bad, mkay? What strategies / methods / tools / etc. what can be done to bypass these SPOFs and make Hadoop redundant, fail-safe, and robust (warning warning)?
source to share
HDFS and Mapreduce are the main components of Hadoop. In earlier versions of Apache Hadoop, Namenode and Jobtracker were SPOF (only one instance can be configured). This issue has been fixed from Hadoop 2.X.
Jobtracker HA.
Jobtracker HA can be achieved by configuring 2 Jobtracker(JT)
instances in Active - Standby
dual node mode . If one JT goes down, a second Jobtracker request will be available to serve the request. Only one jobtracker (Active) will be available to submit the request at a time, the second JT (Standby) will be read-only. Jobtracker HA requires a zookeeper instance, Failure over can be configured as Manaul or Automcatic. Automatic failover requires another process calledFailover Controller (FC)
... In the current release, if JT is active, all work orders will be stopped, but the new job will be automatically submitted to the new JT. This functionality is not available in the current version.
MR2
- second generation mapreduce which uses YARN Resource Manager(RM)
is the main service in YARN, RM can also be configured in active standby mode. RM error does not affect performance / application.
Namenode HA
Namenode HA is something important. Namenode HA can also be configured in active standby mode (maximum 2 namenode instances). Quorum based Journaling
is a widely accepted method that uses zookeeper internally. Only one namenode will be active at a time.
Secondary Namenode(SNN)
is not, Standby Namenode(SN)
and vice versa, SNN has different functionality in Non HA configuration, no SNN is required to configure the Namenode HA as the SN namenode does the audit trail (SNN functionality)
Processes Namenode HA
- Active namenode
- Reserve namenode
- Failover controller: for fencing to avoid split-brain scenario.
- Jounalnodes (minimum 3 instances required): Modification of the namespace will be logged to the Log nodes, and idle reads will be read from there. Only one namenode will be allowed to write at a time to avoid the split-brain problem.
source to share
There are mechanisms High availability
built into Hadoop for a while. "Secondary NameNode", "Backup JobTracker" will serve as a hot backup of their respective copies.
Most of the past "SPOFs" have been eliminated with the latest chaop releases.
This is explained in detail in the following documents.
Hope it helps.
source to share