FileStreaming Principles

I recently worked on a project that includes a lot FileStreaming

that I still haven't touched on.
To get a better understanding of the principles of such methods, I wrote a code that (in theory) loads a file from one dir

to another, and went through it step by step, commenting in my understanding what each step achieves, like this ...

Get fileinfo object from DownloadRequest object

RemoteFileInfo fileInfo = svr.DownloadFile(request);

      

DownloadFile Method in WCF Service

public RemoteFileInfo DownloadFile(DownloadRequest request)
            {
                RemoteFileInfo result = new RemoteFileInfo(); // create empty fileinfo object
                try
                {
                    // set filepath
                    string filePath = System.IO.Path.Combine(request.FilePath , @"\" , request.FileName);
                    System.IO.FileInfo fileInfo = new System.IO.FileInfo(filePath); // get fileinfo from path

                    // check if exists
                    if (!fileInfo.Exists)
                        throw new System.IO.FileNotFoundException("File not found",
                                                                  request.FileName);

                    // open stream
                    System.IO.FileStream stream = new System.IO.FileStream(filePath,
                              System.IO.FileMode.Open, System.IO.FileAccess.Read);

                    // return result 
                    result.FileName = request.FileName;
                    result.Length = fileInfo.Length;
                    result.FileByteStream = stream;
                }
                catch (Exception ex)
                {
                    // do something
                }
                return result;
            }

      

Use returned FileStream from fileinfo to read into new write stream

// set new location for downloaded file
string basePath = System.IO.Path.Combine(@"C:\SST Software\DSC\Compilations\" , compName, @"\");
string serverFileName = System.IO.Path.Combine(basePath, file);
double totalBytesRead = 0.0;

if (!Directory.Exists(basePath))
    Directory.CreateDirectory(basePath);

int chunkSize = 2048;
byte[] buffer = new byte[chunkSize];

// create new write file stream 
using (System.IO.FileStream writeStream = new System.IO.FileStream(serverFileName, FileMode.OpenOrCreate, FileAccess.ReadWrite))
{
     do
     {
        // read bytes from fileinfo stream
        int bytesRead = fileInfo.FileByteStream.Read(buffer, 0, chunkSize);

        totalBytesRead += (double)bytesRead;

        if (bytesRead == 0) break;

        // write bytes to output stream
        writeStream.Write(buffer, 0, bytesRead);
     } while (true);

  // report end
  Console.WriteLine(fileInfo.FileName + " has been written to " + basePath + "  -  Done!");

  writeStream.Close();
}

      

What I was hoping for was any clarification or expansion of what exactly happens in use FileStream

.
I can achieve loading, and now I know what code I need to write to perform such loading, but I would like to know more about why it works. I cannot find "beginner-friendly" or step-by-step explanations on the internet.

What's going on behind the scenes here?

+3


source to share


2 answers


A stream is just an abstraction, basically it works like a pointer within a dataset.

Let's take an example of the string "Hello World!" for example, it's just a set of characters, which are mostly bytes.

As a stream, you can imagine that:

  • Length 12 (possibly more, including ending characters, etc.)
  • Position in the stream.

You are reading the stream by traversing position and requesting data.

Thus, reading the text above can be (in pseudocode) visible like this:

do
    get next byte
    add gotten byte to collection
while not the end of the stream

the entire data is now in the collection

      

Streams are really useful when it comes to accessing data from sources such as the file system or remote computers.

Imagine a file several gigabytes in size, if the OS loaded it all into memory anytime a program wanted to read it (say, a video player), there would be a lot of problems.

What happens instead is that the program requests access to the file and the OS returns a stream; a thread tells the program how much data there is and allows it to access that data.



Depending on the implementation, the OS may load a certain amount of data into memory before accessing it, this is known as a buffer .

Basically, the program just requests the next bit of data, and the OS either gets it from the buffer or from the source (for example, a file on disk).

The same principle applies to streams between different computers, except that requesting the next bit of data can very well involve traveling to a remote computer to request it.

The .NET class FileStream

and base class Stream

are all just layering on Windows systems for working with threads, after all, there is nothing special about them, it's just what you can do with the abstraction that makes them so powerful.

Writing to a stream is exactly the same, but it simply puts data in a buffer ready for the requester to access.


Endless data

As the user pointed out, streams can be used for data of indefinite length.

All streaming operations take time, so reading the stream is usually a block that will wait until data is available.

So you can loop forever while the stream is still open and just wait for the data to show up - an example of this in practice would be live streaming.

+4


source


I have since posted a book - C # 5.0 All-in-One for Dummies - explains everything about all classes, Stream

how they work, what is most appropriate and much more.
Only read for about 30 minutes, there is already such a better understanding.
Great guide!



+1


source







All Articles