Why does Azure WebJob ServiceBus deserialize XML by default?
I have a simple Azure WebJobs ServiceBusTrigger that looks like
public static async void ProcessQueueMessage([ServiceBusTrigger("myqueuename")] String json, TextWriter log) { ... }
Unfortunately it doesn't deserialize JSON as XML (no surprise). I checked the payload and confirmed that it is just a UTF-8 encoded byte array. I have two questions.
- Why is my string assumed to be XML?
- How do I say no, no XML, only a string?
Stack trace:
System.InvalidOperationException: Exception binding parameter 'json' ---> System.Runtime.Serialization.SerializationException: There was an error deserializing the object of type System.String. The input source is not correctly formatted. ---> System.Xml.XmlException: The input source is not correctly formatted.
at System.Xml.XmlExceptionHelper.ThrowXmlException(XmlDictionaryReader reader, String res, String arg1, String arg2, String arg3)
at System.Xml.XmlBufferReader.ReadValue(XmlBinaryNodeType nodeType, ValueHandle value)
at System.Xml.XmlBinaryReader.ReadNode()
at System.Xml.XmlBinaryReader.Read()
at System.Xml.XmlBaseReader.IsStartElement()
at System.Xml.XmlBaseReader.IsStartElement(XmlDictionaryString localName, XmlDictionaryString namespaceUri)
at System.Runtime.Serialization.XmlReaderDelegator.IsStartElement(XmlDictionaryString localname, XmlDictionaryString ns)
at System.Runtime.Serialization.XmlObjectSerializer.IsRootElement(XmlReaderDelegator reader, DataContract contract, XmlDictionaryString name, XmlDictionaryString ns)
at System.Runtime.Serialization.DataContractSerializer.InternalIsStartObject(XmlReaderDelegator reader)
at System.Runtime.Serialization.DataContractSerializer.InternalReadObject(XmlReaderDelegator xmlReader, Boolean verifyObjectName, DataContractResolver dataContractResolver)
at System.Runtime.Serialization.XmlObjectSerializer.ReadObjectHandleExceptions(XmlReaderDelegator reader, Boolean verifyObjectName, DataContractResolver dataContractResolver)
--- End of inner exception stack trace ---
at System.Runtime.Serialization.XmlObjectSerializer.ReadObjectHandleExceptions(XmlReaderDelegator reader, Boolean verifyObjectName, DataContractResolver dataContractResolver)
at System.Runtime.Serialization.DataContractSerializer.ReadObject(XmlDictionaryReader reader, Boolean verifyObjectName)
at Microsoft.ServiceBus.Messaging.DataContractBinarySerializer.ReadObject(XmlDictionaryReader reader, Boolean verifyObjectName)
at System.Runtime.Serialization.XmlObjectSerializer.ReadObject(XmlReader reader, Boolean verifyObjectName)
at System.Runtime.Serialization.XmlObjectSerializer.InternalReadObject(XmlReaderDelegator reader, Boolean verifyObjectName)
at System.Runtime.Serialization.XmlObjectSerializer.InternalReadObject(XmlReaderDelegator reader, Boolean verifyObjectName, DataContractResolver dataContractResolver)
at System.Runtime.Serialization.XmlObjectSerializer.ReadObjectHandleExceptions(XmlReaderDelegator reader, Boolean verifyObjectName, DataContractResolver dataContractResolver)
at System.Runtime.Serialization.XmlObjectSerializer.ReadObject(XmlDictionaryReader reader)
at Microsoft.ServiceBus.Messaging.DataContractBinarySerializer.ReadObject(Stream stream)
at Microsoft.ServiceBus.Messaging.BrokeredMessage.GetBody[T](XmlObjectSerializer serializer)
at Microsoft.ServiceBus.Messaging.BrokeredMessage.GetBody[T]()
at Microsoft.Azure.WebJobs.ServiceBus.Triggers.BrokeredMessageToStringConverter.ConvertAsync(BrokeredMessage input, CancellationToken cancellationToken)
at Microsoft.Azure.WebJobs.ServiceBus.Triggers.ConverterArgumentBindingProvider`1.ConverterArgumentBinding.<BindAsync>d__0.MoveNext()
Edit: The WebJobs documentation suggests that not only what I did ( String
), but the ServiceBusTrigger should deserialize JSON objects automatically. However, if I try to get my POCO, I still get the XML deserialization error. Interestingly, I also get an XML deserialization error if I set the type to Byte [], which should also work.
Edit 2: Stream
doesn't work either. It looks like only BrokeredMessage works for trigger and GetBody is the only way to find String from BrokeredMessage.
source to share
If your payload is String, you have two options:
- Change the type of the parameter to
BrokeredMessage
and then deserialize it yourself - Set the property to a
BrokeredMessage.ContentType
valuetext/plain
(this assumes that you are in control of the code that generates the message).
Except for the weird case String
where you need a content type, the rule of thumb is that Service Bus overhead can only be deserialized to the same object as the payload. This is because of the ServiceBus binary serializer.
source to share
I was able to get the deserialization to work by adding a custom message handler.
An example message handler is available at https://github.com/Azure/azure-webjobs-sdk-samples/tree/master/BasicSamples/MiscOperations
You are setting ContentType to application / json as shown below -
public class CustomMessagingProvider : MessagingProvider
{
private readonly ServiceBusConfiguration _config;
public CustomMessagingProvider(ServiceBusConfiguration config) : base(config)
{
_config = config;
}
public override MessageProcessor CreateMessageProcessor(string entityPath)
{
return new CustomMessageProcessor(_config.MessageOptions);
}
private class CustomMessageProcessor : MessageProcessor
{
public CustomMessageProcessor(OnMessageOptions messageOptions)
: base(messageOptions)
{
}
public override Task<bool> BeginProcessingMessageAsync(BrokeredMessage message, CancellationToken cancellationToken)
{
message.ContentType = "application/json";
return base.BeginProcessingMessageAsync(message, cancellationToken);
}
}
}
Then you install this message processor when you configure the ServiceBusConfiguration in the webjob.
JobHostConfiguration config = new JobHostConfiguration();
ServiceBusConfiguration serviceBusConfig = new ServiceBusConfiguration
{
ConnectionString = _servicesBusConnectionString
};
serviceBusConfig.MessagingProvider = new CustomMessagingProvider(serviceBusConfig);
config.UseServiceBus(serviceBusConfig);
source to share
I found several voodoo while I was researching this issue.
We did a great job with binary serialization, but wanted to see what the body of the message is when we look at how webjobs handle events through the webjobs toolbar. I searched for code from the WebJobs ServiceBus code on github to resolve this issue. I'm still a little confused as to why you should serialize poco to JSON and package it into a memory stream and then set the content type to application / json. My conclusion is that there must be something internally inside it with the Azure Service Bus SDK expecting this. I haven't found this code yet, but in testing it works great.
We first had this case where the object serialized in the webjobs toolbox is just bytes.
After using the below code to post messages to the bus, we get:
We didn't need to change the signature on the consumption side, and now we can see the body of our BrokeredMessage.
public void PersistEvents([ServiceBusTrigger(EventProcessor.CoolTopicName, LoggingSubscription)] EventDto eventDto, TextWriter logger)
Here is the code for the
Azure ServiceBus SDK that fixed it.
And here is my sample code (borrowed from SDK):
string json = JsonConvert.SerializeObject(eventDto); // Using Json.net here
MemoryStream stream = new MemoryStream(Encoding.UTF8.GetBytes(json), writable: false);
BrokeredMessage message = new BrokeredMessage(stream);
message.ContentType = "application/json";
client.Send(message);
source to share
In response to # 2, a hacky workaround is to get BrokeredMessage
from ServiceBusTrigger
and call message.ToBody<Stream>()
. Then you can use normal stuff to turn the byte stream into a string.
An example of getting a descriptive JSON payload into an object:
public static async void ProcessQueueMessage([ServiceBusTrigger("my-queue")] BrokeredMessage message)
{
var stream = message.ToBody();
using (var streamReader = new StreamReader(stream, Encoding.UTF8)
{
var json = await streamReader.ReadToEndAsync();
var deserialized = JsonConvert.DeserializeObject(json);
}
}
Note. I am still interested in the best answer if anyone has one. In an ideal world, I could provide the ServiceBusTrigger with its own deserializer (based on JSON.net or just a String descriptor), but I don't know how to do this if possible.
source to share