Communication between Azure Service Fabric services
I currently have a single Service Fabric application consisting of multiple services. What I am trying to achieve is a Queuing mechanism , so one service can post a message to a queue and another service can receive messages from one queue .
The following does not work (there is nothing to deactivate for the Listener service):
PublisherService
:
protected override async Task RunAsync(CancellationToken cancellationToken)
{
var myQueue = await StateManager.GetOrAddAsync<IReliableQueue<string>>("fooQueue");
while (true)
{
cancellationToken.ThrowIfCancellationRequested();
using (var tx = this.StateManager.CreateTransaction())
{
// Put some message in the queue
await myQueue.EnqueueAsync(tx, "Foobar");
await tx.CommitAsync();
}
await Task.Delay(TimeSpan.FromSeconds(1), cancellationToken);
}
}
ListenerService
:
protected override async Task RunAsync(CancellationToken cancellationToken)
{
var myQueue = await StateManager.GetOrAddAsync<IReliableQueue<string>>("fooQueue");
while (true)
{
cancellationToken.ThrowIfCancellationRequested();
using (var tx = this.StateManager.CreateTransaction())
{
var result = await myQueue.TryDequeueAsync(tx);
if (result.HasValue)
{
ServiceEventSource.Current.ServiceMessage(this.Context, "New message receieved: {0}", result.Value.ToString());
}
await tx.CommitAsync();
}
await Task.Delay(TimeSpan.FromSeconds(1), cancellationToken);
}
}
It looks like the queue size is limited to one service . This is not a limitation stated in the documentation.
So my questions are:
- Is this actually some kind of undocumented limitation?
- or is there something wrong in the code above?
- How can I accomplish the above scenario (one service adds messages to a queue, another service receives messages from one queue)?
Obviously I could use Azure Service Bus, but I can't for several reasons:
- in my real world scenario, I will have multiple queues (variable number) so this would require creating Service Bus queues on demand (which is not a fast operation)
- adds a dependency to another Azure service (therefore increases the chance of failure for the entire system)
- costs more
- more complex deployment
- and etc.
source to share
ReliableQueues are local to the service, yes, because its purpose is to maintain state for that particular service. This state is replicated to other instances. It's just like normal System.Collections.Generic.Queue<T>
in .Net.
For a low-cost solution, perhaps you can use Azure storage queues . Yes, it adds dependency, but it is highly available. This is a compromise that you can accept or accept.
On the other hand, think out of the box:
Create a stateful service with multiple ReliableQueues and disclosure methods that other services can call using a remote access connection, for example:
class QueuingService
{
void AddToQueue<T>(string queuename, T input) { .. }
void DeQueue(string queuename) { .. }
}
This creates a dependency, of course, but it has all the security mechanisms that the Service Fabric provides and is not expensive to you. But then again, you are building a poor service bus / azure storage queue yourself.
No about the docs, it doesn't say so many words that the reliable queue is bound to 1 service, but it depends on how you interpret this
Service Fabric offers a stateful programming model available to .NET developers through Reliable Collections. In particular, Service Fabric provides robust dictionary and robust queue classes. When you use these classes, your state (my interpretation: service state) is stripped (for scalability), replicated (for accessibility), and translated internally (for ACID semantics).
source to share
If you add a retry error pattern for all the calling code, you don't need a queue between your calls, see https://docs.microsoft.com/en-us/azure/service-fabric/service-fabric-reliable-services-communication
The relevant part of the link is here:
The exception handler is responsible for determining what action to take when an exception is thrown. Exceptions are classified as repetitive and non-retry. Exceptions are simply thrown back to the caller. repeated exceptions are further categorized as transient and non-transient. Temporary exceptions are ones that can simply be repeated without re-resolving the service endpoint address. These will include transient network issues or service error responses other than those indicating that the service endpoint address does not exist. Continuous exceptions are those that require re-resolution of the service endpoint address. These include exceptions indicating that the service endpoint cannot be reached, indicatingthat the service has moved to a different node.
source to share