Flink error - org.apache.hadoop.ipc.RemoteException: IPC version 9 server cannot communicate with client version 4

I am trying to get flink up and running using a file from HDFS. I created a dataset like this:

DataSource<Tuple2<LongWritable, Text>> visits = env.readHadoopFile(new TextInputFormat(), LongWritable.class,Text.class, Config.pathToVisits());

      

I am using the latest flink - 0.9.0-milestone-1-hadoop1 (I also tried with 0.9.0-milestone-1)

whereas my Hadoop version is 2.6.0

But I am getting the following exception when I try to complete the job. I was looking for a similar issue and it has to do with version incompatibility between client and hdfs.

Exception in thread "main" org.apache.hadoop.ipc.RemoteException: Server IPC version 9 cannot communicate with client version 4
at org.apache.hadoop.ipc.Client.call(Client.java:1113)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
at com.sun.proxy.$Proxy5.getProtocolVersion(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

      

Could you please let me know what changes should be made to my pump to point to the correct version of Hadoop / HDFS? or changes elsewhere? Or do I need to downgrade hadoop installation?

+3


source to share


1 answer


Have you tried building Hadoop-2 from Flink? Check out the download page . There is an assembly called flink-0.9.0-milestone-1-bin-hadoop2.tgz

that should work with Hadoop 2.



+3


source







All Articles