Azure Service Bus - delete a specific message

I am developing a system that will include a lot of data synchronization, broken down into small tasks. I add each little task as a job / message to the Azure Service Bus queue.

I have X number of worker roles, then check queues and process data.

I don't expect there to be many messages in the queue, because the goal is to process the message, complete it, and then re-add the same message, but scheduled for X minutes. This will give me a loop to continue processing these tasks.

The great thing about the Azure functionality is that they handle all the server stuff for you, but the downside is that it can sometimes be difficult to debug or manipulate data.

What I want to do is list the queued messages (which I made with PeekBatch) in the web interface. Then I want to be able to select some / all messages and delete them.

I could do this if there is an error in the code and I want to stop messages of a certain type to stop.

After that, I will have the option to re-add posts from the web page. I might want my worker roles and messages to complete the task faster (or slow them down) or re-add deleted messages.

So the question is, how can I actually select a specific message from the queue and then delete it? From what I can see, there is no obvious way to do this, and if possible it will require some kind of workaround. This sounds a little strange to me.

Edit:

I have something that works, but it doesn't really seem like a great solution:

    public void DeleteMessages(List<long> messageIds)
    {
        foreach (var msg in Client.ReceiveBatch(100))
        {
            if (messageIds.Contains(msg.SequenceNumber))
                msg.Complete(); // Deletes the message
            else
                msg.Abandon(); // Puts it back in the queue
        }
    }

      

This will become less and less efficient the larger the queue, but at least it stops all activity while the delete call is in progress and deletes the specified messages.

It will also only delete messages that are ready to be processed. Messages in the future will be ignored, so I added the ability to add Sleep messages to stop the queue from processing until my messages are β€œready” and I can delete them.

I have been informed by Microsoft that they are currently working on an API for deleting specific messages that should be available in a few months. Until then, this is all about workarounds.

June update:

There is no update yet from Microsoft on this issue and the above method was less than ideal. Now I have changed my code to:

The object that I entered in the post has a new property:

Guid? MessageId { get; set; }

      

Note that it is zero. For backward compatibility only.

When I want to delete a message, I add my MessageId to the "DeletedMessage" database table.

When it comes to processing a message, I look in the DeletedMessage table for the corresponding Guid, and if it finds one, I'll just end () the message without doing the usual processing.

This works well, but it's a small overhead. Unless you are dealing with a huge amount of messages, this is not a big problem.

Also note that I did this originally with SequenceNumber, but (strangely) the SequenceNumber changes between peeking and retrieve the message! This prevents the idea from working unless you use your own ID as above.

+3


source to share


2 answers


You can create your own queue cleanup program that has a message listener that peeps in and either refuses or commits a message depending on some message criteria. This could be a controller action if you also anticipate the need for purge in real production environments.



0


source


From my experiments, it is not possible to remove an active message from the message queue.

Delayed messages can be deleted by calling Receive(sequenceNumber)

and then calling Complete()

.

Scheduled messages can be deleted by calling CancelScheduledMessageAsync(sequenceNumber)

.



For active messages, you need to make sure they are moved to the dead letter queue at some point.

Dead letter queued messages can then be subsequently deleted or resent.

0


source







All Articles