Clever Geek Handbook

Improving Akka throughput

We are thinking about using Akka to communicate with the client server and trying to compare data transfers. We are currently trying to send a million messages, where each message is a case class with 8 string fields.

At the moment we are trying to get acceptable performance. We see about 600KB / s transfer rates and idle processors on the client and server, so something is going wrong. Maybe this is our network configuration.

This is our akka config

Server {
  akka {
    extensions = ["akka.contrib.pattern.ClusterReceptionistExtension"]
    loggers = ["akka.event.slf4j.Slf4jLogger"]
    loglevel = "DEBUG"
    logging-filter = "akka.event.slf4j.Slf4jLoggingFilter"
    log-dead-letters = 10
    log-dead-letters-during-shutdown = on
    actor {
      provider = "akka.cluster.ClusterActorRefProvider"
    }
    remote {
      enabled-transports = ["akka.remote.netty.tcp"]
      netty.tcp {
        hostname = "instance.myserver.com"
        port = 2553
        maximum-frame-size = 1000 MiB
        send-buffer-size = 2000 MiB
        receive-buffer-size = 2000 MiB
      }
    }
    cluster {
      seed-nodes = ["akka.tcp://server@instance.myserver.com:2553"]

      roles = [master]
    }
    contrib.cluster.receptionist {
      name = receptionist
      role = "master"
      number-of-contacts = 3
      response-tunnel-receive-timeout = 300s
    }
  }

}

Client {
  akka {
    loggers = ["akka.event.slf4j.Slf4jLogger"]
    loglevel = "DEBUG"
    logging-filter = "akka.event.slf4j.Slf4jLoggingFilter"
    actor {
      provider = "akka.remote.RemoteActorRefProvider"
    }
    remote {
      enabled-transports = ["akka.remote.netty.tcp"]
      netty.tcp {
        hostname = "127.0.0.1"
        port = 0
        maximum-frame-size = 1000 MiB
        send-buffer-size = 2000 MiB
        receive-buffer-size = 2000 MiB
      }
    }
    cluster-client {
      initial-contacts = ["akka.tcp://server@instance.myserver.com:2553/user/receptionist"]
      establishing-get-contacts-interval = "10s"
      refresh-contacts-interval = "10s"
    }
  }

}

UPDATE:

In the end, despite the discussion of serialization (see below), we just converted our payloads to use byte arrays, so serialization won't affect the tests. We found that on a 24GB drum i7 core using jeroMQ (i.e. ZeroMQ reimplemented in java so still not the fastest) we saw consistently around 200k msgs / sec, or around 20 MB / sec. and on raw akka (that is, without the zeroMQ plugin) we saw about 10 thousand msgs / sec, or slightly less than 1 MB / s. Trying akka + zeroMQ made the job worse.

+3


source to share


2 answers


To make it easier to deal with deletion, akka uses Java serialization to serialize messages, this is not something you usually use in production because it is not very fast and does not handle message versioning.

What you have to do is use kryo or protobuf for serialization and you should be able to get much better numbers.



You can read about how there are several links to the available serializers here at the bottom of the page: http://doc.akka.io/docs/akka/current/scala/serialization.html

+4


source


It's good that we found:

Using our client server model, we were able to get information about <3000 msg / sec without any action, which was unacceptable for us, but we were really confused about what was happening because we were not able to maximize the processor.

So we went back to the original akka code and found a comparison sample there:

sample.remote.benchmark.Receiver

sample.remote.benchmark.Sender



These are two classes that use akka remote to ping-pong by the very bunch of messages from two jvms on one machine. Using this benchmark, we were able to get about 10 ~ 15k msg / sec on a Corei7 with 24g using about 50% CPU. We found that adjusting the dispatchers and threads allocated for the netty made a difference, but only marginally. Using queries instead of prompts made it a little slower, but not by much. Using a balancing pool of many actors to send and receive has made it significantly slower.

For comparison, we ran a similar benchmark using jeromq and managed to get around 100K msg / sec using 100% CPU (same payload, same computer).

We also hacked Sender and Receivers to use the Akka zeromq extension to pass messages directly from one jvm to another, bypassing akka remote together. In this test, we found that we could send and receive quickly to start at around 100k Msg / sec, but that performance deteriorated quickly. Upon further examination, the zeromq and chord strings do not play together. We could probably tweak the performance by using smarter how zeromq and akka interacted with each other, but at this point we decided it would be better to go with the raw zeromq.

Conclusion: Don't use akka if you care about serializing large amounts of data quickly over the wire.

0


source






More articles:


All Articles