NullPointerException in Avro ReflectDatumWriter
I have a particular problem with Avro serialization of Java objects. I have a POJO generated from xsd schemas which I then try to serialize using avro to post on kafka theme. Some of the xmlElements are optional
The test message looks like this:
@XmlRootElement(name = "message")
public class Testmessage {
@XmlElement
public String id;
@XmlElement
public String name;
public Testmessage(String id, String name) {
this.id = id;
this.name = name;
}
public Testmessage() { }
@Override
public String toString() {
return "Message{" +
"id='" + id + '\'' +
", name=" + name +
'}';
}
}
And the serialization and posting method by topic:
public void sendMessage(Testmessage msg) throws Exception{
DatumWriter<Testmessage> writer = new ReflectDatumWriter<Testmessage>(Testmessage.class);
ByteArrayOutputStream os = new ByteArrayOutputStream();
Encoder encoder = EncoderFactory.get().binaryEncoder(os, null);
writer.write(msg, encoder);
encoder.flush();
os.close();
KeyedMessage<String, byte[]> data = new KeyedMessage<String, byte[]>(TOPIC_NAME, os.toByteArray());
producer.send(data);
}
When I submit both fields, everything works as expected. If I remove one of the fields, or leave it alone, I get the NPE from the record.
java.lang.NullPointerException: in Testmessage in string null of string in field id of Testmessage
at org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:145)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:58)
Any ideas? or point me in the right direction
Thank!
source to share
Looks like I ended up managing this myself after posting, a couple of days after reading interns later
ReflectData reflectData = ReflectData.AllowNull.get();
Schema schema = reflectData.getSchema(Testmessage.class);
DatumWriter<Testmessage> writer = new ReflectDatumWriter<Testmessage>(schema);
Appears to allow the use of zeros rather happily.
To my next mistake! which the
org.apache.avro.UnresolvedUnionException: Not in union ["null",{"type":"record","name":"XMLGregorianCalendar","namespace":"javax.xml.datatype","fields":[]}]: 2014-10-22
at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:604)
at org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:151)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:71)
at org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:143)
at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:114)
source to share
Avro does not support all data types. Date and time support was added recently ( https://issues.apache.org/jira/browse/AVRO-739 )
So it might be because XMLGregorianCalendar is not supported as a datatype in Avro. Could you try to use some other datatype (it is recommended to use String first, before moving on to a more complex datatype)?
source to share