How to determine what buffer size to use when using parBuffer in Haskell
documentaion for parBuffer
says
parBuffer :: Int -> Strategy a -> Strategy [a]
Like evalBuffer, but checks the elements of the list in parallel when they are pressed into the buffer.
evalBuffer :: Int -> Strategy a -> Strategy [a]
evalBuffer is a pull buffer strategy combinator for (lazy) lists. evalBuffer is not as compositional as the type suggests. In fact, it evaluates the elements of the list in at least weak normal head form, apart from the strategy argument r0. "
The first part of my question is, does this mean that it parBuffer
also ignores its strategy argument? Also, why does it even provide a strategy argument if it just ignores it?
The second part of my question is how to determine what buffer size to use with parBuffer
? If parBuffer
always maintains the buffer size n
, what is the difference between n=1
and n=10
apart from other sparks should be kept in memory? I thought it would be wise to choose n=<number of threads>"
, but I don't know how it would improve anything because it parBuffer
will create a spark whenever the spark is consumed no matter n=1
or n=4
.
source to share
First answer: parBuffer
Takes a value Strategy
to evaluate one item in the list and returns Strategy
to evaluate the entire list. So yes, it evaluates the list in parallel. But he still needs to know how to evaluate each item. (Do you want normal shape or normal shape for a weak head or ...?) So it doesn't ignore the first argument.
Second ... Rate this? I suspect the answer depends on how much work each spark does, how much data you have, etc.
source to share