Generating a recursive sequence
I am trying to create a sequence of slices by recursively reading data from my data source.
let rec read stream startFrom (conn : IEventStoreConnection) =
seq {
let size = 10000
let slice = conn.ReadStreamEventsForwardAsync(stream, startFrom, size, true).Result
Console.WriteLine("Events in slice: " + slice.Events.Count().ToString())
if (slice.IsEndOfStream) then
yield slice
else
yield! read stream (startFrom + 1 * size) conn
}
let slices = read "stream-name" 0 conn
Console.WriteLine("TOTAL slices in sequence: " + (Seq.length slices).ToString());
I expected the sequence to have three elements, but it seems to only return one, the last one.
Events in slice: 10000
Events in slice: 10000
Events in slice: 4501
TOTAL slices in sequence: 1
Why can't I get the result of the recursive call? What is the subject of the review?
+3
source to share
2 answers
As John already told you, you should also issue other slices:
let rec read stream startFrom (conn : IEventStoreConnection) =
seq {
let size = 10000
let slice = conn.ReadStreamEventsForwardAsync(stream, startFrom, size, true).Result
Console.WriteLine("Events in slice: " + slice.Events.Count().ToString())
if (slice.IsEndOfStream)
then
yield slice
else
yield slice // here
yield! read stream (startFrom + 1 * size) conn
}
I think you can clean up this a bit by using internal functions:
let read (conn : IEventStoreConnection) stream startFrom =
let size = 10000
let rec loop startFrom =
seq {
let slice = conn.ReadStreamEventsForwardAsync(stream, startFrom, size, true).Result
Console.WriteLine("Events in slice: " + slice.Events.Count().ToString())
if (slice.IsEndOfStream)
then
yield slice
else
yield slice // here
yield! loop (startFrom + 1 * size)
}
loop startFrom
+4
source to share
For an alternative adoption of this (which does not attempt to encapsulate sliced ββloading in this way, preferring to manage this aspect separately in order to parallelize the "play" of events, be theta with NES or GES) see the load / deserialization loop in FunDomain .
NB all the code in FunDomain is really just FsUno.Prod refactored a lot.
+1
source to share