Clever Geek Handbook

Concurrent priority queue in redis?

I would like to implement a parallel priority queue in Redis, with multiple processes on different machines, adding items (with scores), and several other processes that lay out those items, at first with minimal count.

A simple queue can be implemented using LPUSH and RPOP.

Using ZSET I can add elements using ZADD and put them in ZRANGE and ZREM if there is only one reader.

For a couple of readers, I think I need something like ZPOP that combines ZRANGE and ZREM in one atomic operation. Otherwise, two readers could receive the same item from ZRANGE before it can ZREM. Retrying if ZREM returns 0 will work, but not desirable.

Is there a way to do this using the current Redis commands? Is there a reason this hasn't been added to Redis yet? It looks like it will be a pretty simple command to implement.

+3


source to share


2 answers


You can guarantee atomicity if you use a Lua script that does ZRANGE and ZREM, or with a MULTI / EXEC block. This will prevent multiple employees from interfering with each other.



My guess is that ZPOP was not put in the first place because it is not a common use case and can be scripted easily if needed.

+4


source


you can use redis command: watch

WATCH zset
element = ZRANGE zset 0 0
MULTI
ZREM zset element
EXEC


If exec doesn't work (returns a null response), just repeat these commands.

+1


source






More articles:


All Articles