Increase counters in azure table
I am planning on using Azure Tables to store various application related counters. My problem is that the counters are increasing from different client instances of the application.
It looks like a simple task on the surface, but looking at it is quite difficult.
I hope someone can share some wisdom to help make the problem more manageable.
Thank!
... Short version:
- The client contacts the web role.
- The web role updates the static variable using a function
System.Threading.Interlocked
. This avoids multiple threads on the server trying to update the same number. - Each webrole has a background thread that, by default, writes back to the count table, which includes the instance name as part of its PartitionKey / RowKey. This avoids two instances trying to update the same number at the same time.
Instead of keeping a counter, why don't you keep one line at a time. A collection of data to find out the value of the counter.
The hybrid approach will keep hitting coincidental. Keep doing aggregates and saving. Then the old beat data is deleted / deleted.
With the introduction of Redis, Cache Redis can be used for this task and redis is atomic incremented ( http://redis.io/commands/INCR ).
Depending on the number of counters and the refresh rate, Azure table storage might not be the best way to go. First, table storage does not provide commit or transactional support or the ability to update a value on the server without first getting it. Thus, you may run into a situation where your update fails because another instance has updated a row while trying to update the same row.
If the volume and frequency of updates is repeated frequently when you re-run updates, you may need to consider SQL Azure, instead of which updates can be made atomic on the server side. (something like UPDATE Abc SET Value = Value + 1 WHERE ....).