What does "round robin" mean in Apache Spark?
Spark is the DAG execution engine. Aren't cyclicals and DAGs opposite? It is surprisingly difficult to find an answer to this seeming contradiction.
As you can see here: Understanding Your Apache Spark Application Through Rendering , you can visualize the execution DAG using the Spark interface. However, none of the examples on this page show cyclic data flow. In the following figure, you can see one of these examples.
An example of a DAG execution of a DAG
Could these iterations (cyclical data streams) be outside the graph? I read in MAPR that "Every Spark job creates a DAG of the steps of a task that needs to run on a cluster." Then, perhaps a cyclical data flow occurs between DAGs (jobs).
Thank.
source to share
Ok, looks like it was a typo or something in the documentation. As of today, we can find this in the Spark page :
Apache Spark has an improved DAG execution engine that supports acyclic data flow and in-memory computation.
source to share