IllegalAccessError for RequestHedgingRMFailoverProxyProvider when running Apache Twill app in Hop cluster after HDP update
I am trying to run Apache Twill application from hadoop cluster, cluster recently upgraded from HDP 2.2 to HDP 2.5, but I am getting llegalAccessError for RequestHedgingRMFailoverProxyProvider class. This class is part of the org.apache.hadoop.yarn.client package. I am getting this error in Application Master. The running status goes straight to the "not running state" immediately after the "accepted state".
Exception in thread "Hadoop22YarnAMClient STARTING" Exception in thread "YarnAMClientService STARTING" java.lang.IllegalAccessError: tried to access method org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider.getProxyInternal()Ljava/lang/Object; from class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider
at org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider.init(RequestHedgingRMFailoverProxyProvider.java:75)
at org.apache.hadoop.yarn.client.RMProxy.createRMFailoverProxyProvider(RMProxy.java:163)
at org.apache.hadoop.yarn.client.RMProxy.createRMProxy(RMProxy.java:93)
at org.apache.hadoop.yarn.client.ClientRMProxy.createRMProxy(ClientRMProxy.java:72)
at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.serviceStart(AMRMClientImpl.java:186)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at org.apache.twill.internal.yarn.Hadoop21YarnAMClient.startUp(Hadoop21YarnAMClient.java:77)
at com.google.common.util.concurrent.AbstractIdleService$1$1.run(AbstractIdleService.java:43)
at java.lang.Thread.run(Thread.java:745)
com.google.common.util.concurrent.ExecutionError: java.lang.IllegalAccessError: tried to access method org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider.getProxyInternal()Ljava/lang/Object; from class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider
at com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1008)
at com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1001)
at com.google.common.util.concurrent.AbstractService.startAndWait(AbstractService.java:220)
at com.google.common.util.concurrent.AbstractIdleService.startAndWait(AbstractIdleService.java:106)
at org.apache.twill.internal.appmaster.ApplicationMasterMain$YarnAMClientService.startUp(ApplicationMasterMain.java:221)
at com.google.common.util.concurrent.AbstractIdleService$1$1.run(AbstractIdleService.java:43)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalAccessError: tried to access method org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider.getProxyInternal()Ljava/lang/Object; from class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider
at org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider.init(RequestHedgingRMFailoverProxyProvider.java:75)
at org.apache.hadoop.yarn.client.RMProxy.createRMFailoverProxyProvider(RMProxy.java:163)
at org.apache.hadoop.yarn.client.RMProxy.createRMProxy(RMProxy.java:93)
at org.apache.hadoop.yarn.client.ClientRMProxy.createRMProxy(ClientRMProxy.java:72)
at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.serviceStart(AMRMClientImpl.java:186)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at org.apache.twill.internal.yarn.Hadoop21YarnAMClient.startUp(Hadoop21YarnAMClient.java:77)
... 2 more
source to share
In general, when you see it IllegalAccessError
, it means that you have a runtime incompatibility between compiled and executable code. In this case, the
getProxyInternal()
method is ConfiguredRMFailoverProxyProvider
now private. You need to recompile your client code and / or use the updated hadoop client libraries to connect to your cluster.
source to share