Casting mmted ByteString for other types?
I realize this might be a rather heretical question, but I am wondering if I can mount a data file via System.IO.Posix.MMap and then the cast
resulting ByteString into a strict array of some other type? For example. if I know the file contains doubles, can I somehow get this mmapped data into a UArr Double so that I can do sumU etc. and so that the virtual memory system takes care of the IO for me? This is essentially how I deal with multi GB datasets in my C ++ code. Alternative more idiomatic ways to do this were also appreciated, thanks!
Top bonus points for paths that I can also do multi-core processing :-) Not that I require it or anything else.
source to share
I don't think it's safe. UArr is Haskell heap, allocated unpinned memory, GC will move it. ByteStrings (and mmapped) are ForeignPtrs for memory pinning. They are different objects in the runtime system.
You will need to copy to make this safe if you change the base type from ForeignPtr to the Haskell value 'a'.
source to share
I'm afraid I don't know how to tell the difference ByteString
from UArr T
, but I would like to gain some "extra points" by suggesting you take a look at Data Parallel Haskell ; from the problem you described, it could be right on your street.
source to share
You probably need Foreign.Marshal here, and especially Foreign.Marshal.Array . It was designed to do just that.
source to share