Scala pipelines - DSL to create a DAG workflow
I am wondering about the current libraries for Scala and Akka that would allow me to elegantly build a sequence pipeline.
In my case, the workflow is just a DAG of operations, so the Actors / Akka feel good.
My question is the best approach? There are Libs like reactive streams that allow for really nifty pipeline composition, but they seem to be very write-focused.
My use case is a workflow passing messages between them. The future composition is fine, but the syntax becomes cumbersome after a while. Maybe something better with storytelling and formless.
What are the approaches and tools to build DSL for pipelines of computation steps using message passing?
source to share
If your pipeline is very similar to method chaining, use method chaining!
There is no point in making the decision more difficult than necessary; if it's well modeled with method call chaining, just use that. (Or functions you can create.)
If you need something a little more complex, but you don't really need messaging, you might need something like AsyncFP or Scala.Rx .
If you need a multi-core solution, but have stretch lines that look like method calls, then there is a chaining of method calls inside one stop. You can use Akka streams for this without worrying about over-costing a useful computation factor.
source to share
While it's too early to develop (prior to version 1.0 from the time of writing), you should take a look at akka-streams which is exactly this way of describing a compute graph, and then run it asynchronously.
source to share