Clever Geek Handbook

Multiple streams written in one file

I would like to know if we can use multiple streams to write binary data to the same file.

FILE *fd = openfile("test");
int SIZE = 1000000000;
int * table = sizeof(sizeof(int) * SIZE);
// .. filling the table
fwrite(table, sizeof(*table), SIZE, fd);

so I wonder if I can use streams and each stream calls fssek to look for a different write location in the same file.

Any idea?

+3


source to share


4 answers


fwrite should be thread safe, but you need a mutex anyway, because you want lookup and writing is atomic. Depending on your platform, you might have a write function that takes an offset, or you might open the file on each stream. Best option if you have everything in memory, anyway as your code suggests, just fill one big array for each thread and then write that when done.



+1


source


Though fread()

and fwrite()

are thread, flux buffer is submitted FILE*

, it is not. Thus, you can have multiple threads accessing the same file, but not through the same one FILE*

- each thread must have its own, and the file they refer to must be shared, depending on the OS.



An alternative and possibly simpler approach is to use a memory-mapped file , so that each thread treats the file as shared memory, and you let the OS be associated with file I / O. This has a significant advantage over regular file I / O as it is truly random access, so you don't have to worry about fseek()

sequential reads / writes, etc.

+1


source


fseek and fwrite are both thread safe, so you can use them without additional synchronization.

0


source


Let each thread open the file and make sure they write to different positions, finally, have each thread close the file and execute it.

Update:

This works on IX'ish systems, at least.

0


source






More articles:


All Articles