Received unexpected output in MRUnit
I am getting the following MRUnit error:
ERROR mrunit.TestDriver: Unexpected output received (60, mrdp.MyCustomClass@73207f36 )
ERROR mrunit.TestDriver: Missing expected result (60, mrdp.MyCustomClass@6f73cf45 ) at position 0
I have created MyCustomClass
which implements Writable
and has 4 int attributes. This is the output value of my Mapper.
Below is the MRUnit benchmark code for matching:
@Test
public void testMapper() throws IOException {
MyCustomClass result = new MyCustomClass();
result.setAttr1(1);
result.setAttr2(0);
result.setAttr3(0);
result.setAttr4(0);
mapDriver.withInput(new LongWritable(1), new Text("60,5596,1,256"));
mapDriver.addOutput(new Text("60"), result);
mapDriver.runTest();
}
My Mapper should call it setter setAttr1(1)
when you find "1" in new Text("60,5596,1,256")
here above.
How can I check this result with a custom class (with multiple attributes)? The job is done successfully, I just don't know how to make MRUnit work.
$ hadoop fs -cat patterns/minmaxcount/outuserprefs/part*
23 mrdp.MyCustomClass@4cf15f6c
60 mrdp.MyCustomClass@4cf15f6c
source to share
You will need to redefine equals()
and hascode()
to your custom classes if you want to check equality. If you don't, there is no way to test for "semantic equality". The Object
default methods are used . This is what you are facing. For further discussion see Why would I override equals and hashCode methods in Java?
Below is a simple JUnit test using a custom class CustomClass
. I have commented equals
and hashcode
. If you run the test, it will fail, with a similar message to what you are getting. If you delete the comments and run them, they will pass.
import static org.junit.Assert.*;
import org.junit.Test;
public class CustomClass {
String firstName;
String lastName;
public void setFirstName(String firstName) { this.firstName = firstName; }
public void setLastName(String lastName) { this.lastName = lastName; }
@Test
public void testEqaulity() {
CustomClass clazz1 = new CustomClass();
clazz1.setFirstName("Stack");
clazz1.setLastName("Overflow");
CustomClass clazz2 = new CustomClass();
clazz2.setFirstName("Stack");
clazz2.setLastName("Overflow");
assertEquals(clazz1, clazz2);
}
/*
@Override
public int hashCode() {
final int prime = 31;
int result = 1;
result = prime * result
+ ((firstName == null) ? 0 : firstName.hashCode());
result = prime * result
+ ((lastName == null) ? 0 : lastName.hashCode());
return result;
}
@Override
public boolean equals(Object obj) {
if (this == obj)
return true;
if (obj == null)
return false;
if (getClass() != obj.getClass())
return false;
CustomClass other = (CustomClass) obj;
if (firstName == null) {
if (other.firstName != null)
return false;
} else if (!firstName.equals(other.firstName))
return false;
if (lastName == null) {
if (other.lastName != null)
return false;
} else if (!lastName.equals(other.lastName))
return false;
return true;
}
*/
}
If you have no experience or knowledge of implementing these methods, most IDEs have the ability to create them for you.
- Eclipse: right click on class -> Source -> Generate equals and hashcode
- Netbeans: Right Click on Source Editor -> Paste Code -> equals () hashcode ()
In both cases, you need to select the properties you want to include (check) in equals and hashcode. These are the only two IDEs I use :-)
source to share