Distributed sequence generator (long) in Java. Can anyone confirm if this project is correct?

Requirement . Sequence generator (Long) that will work in a distributed environment (multiple JVMs). Each JVM will run multiple threads.

Solution . We have a centralized persistent key store. But we don't want to make a remote call for every incoming request, so we thought about fetching a batch of sequence IDs from this centralized keystore and store in the local JVM and then use it.

Key Value Store

: This is our centralized key store where we store an object SEQUENCE_ID

with a Long

value. This key data store has the ability to manage concurrent updates via version number.


: Performs the following operations:

  • Get current value for SEQUENCE_ID

    from keystore
  • Add batch size to the resulting value
  • Update new value for SEQUENCE_ID

Multiple threads might be trying to do this, so all these 3 steps will be done as one atomic job. We use the version number feature of this keystore to manage these side-by-side updates.


: a queue-based data structure that will hold a batch of sequence IDs.


: An observer (implemented through the Observer design pattern) that can check if the size has dropped SequenceHolder

to a threshold, it will use BatchRetriever

to fetch the next batch.

Please rate if anyone can confirm this design and suggest a better one.

~ NN


source to share

1 answer

These are good approaches.

A simpler solution might be to use shared memory. This has the performance of AtomicLong and is shared across processes on the same machine.

import net.openhft.lang.io.DirectBytes;
import net.openhft.lang.io.MappedStore;

import java.io.File;
import java.io.IOException;
import java.nio.channels.FileChannel;

public class CounterExampleMain {
    static volatile long id;

    public static void main(String... ignored) throws IOException {
        int counters = 128;
        int repeats = 100000;

        File file = new File(System.getProperty("java.io.tmpdir") + "/counters");
        MappedStore ms = new MappedStore(file, FileChannel.MapMode.READ_WRITE, counters * 8);
        DirectBytes slice = ms.bytes();

        long start = System.nanoTime();
        for (int j = 0; j < repeats; j++) {
            for (int i = 0; i < counters; i++) {
                id = slice.addAtomicLong(i * 8, 1);
        long time = System.nanoTime() - start;
        System.out.printf("Took %.3f second to increment %,d counters, %,d times, last id=%,d%n",
                time / 1e9, counters, repeats, id);


Every time I run it on my laptop I get

Took 0.252 second to increment 128 counters, 100,000 times, last id=100,000
Took 0.267 second to increment 128 counters, 100,000 times, last id=200,000
Took 0.255 second to increment 128 counters, 100,000 times, last id=300,000


As you can see, it is really cheap, averaging ~ 25 ns per step and persisting between program runs. It is also thread safe and can be used in conjunction with the JVM.

BTW in a happy example where multiple threads are updating the same counters, I would expect closer to 50ns.

The library I used was





All Articles