MRUnit - not working as expected
I only have a mapping class that creates files for bulk upload in HBase and I wrote MRUnit to test the Unit.
Although the expected result and the received results are the same, MRUnit does not work with the message "No expected result" .
Expected Result: (4b 65 79 31, {"totalColumns":1,"families":{"default":[{"timestamp":9223372036854775807,"qualifier":"default","vlen":6}]},"row":"Key1"})
Actual output: (4b 65 79 31, {"totalColumns":1,"families":{"default":[{"timestamp":9223372036854775807,"qualifier":"default","vlen":6}]},"row":"Key1"})
Where am I going wrong? Is this a problem with ImmutableBytesWritable or Put?
source to share
So your failure is that MRUnit uses methods equals
and hashCode
to compare objects of expected and actual output (HashMap method is used in method org.apache.hadoop.mrunit.TestDriver.buildPositionMap(List<Pair<K2, V2>>)
):
- ImmutableBytesWritable - Implements the method
equals
andhashCode
- Put - does not implement the method,
equals
orhashCode
Thus, the Path is causing the problem. The only way to get around this is to manually check the driver's output (use the method run
instead of the driver run
) and compare the list of results with what you expect using the method compareTo
(which exists for both classes).
source to share
I had this problem and solved it by setting a comparator for key and value. Code below for BytesWritable value.
ReduceDriver reduceDrive = ReduceDriver.newReduceDriver(<<ReducerInstance>>);
reduceDriver.setValueComparator(new Comparator<BytesWritable>(){
@Override
public int compare(BytesWritable o1, BytesWritable o2) {
return o1.compareTo(o2);
}
});
source to share