Using multer with inmemory option and streaming in mongodb gridfs
The following piece of code works as expected: it reads a file that was uploaded with multer [ https://github.com/expressjs/multer] and feeds it to gridfs, Im also able to wipe this out from gridfs as well.
var target = gfs.createWriteStream({
filename: fileItem.originalname,
mode: 'w',
metadata: metaData
});
fs.createReadStream(tempFile)
.on('end', function () {
console.log('File read and stored');
})
.on('error', function (err) {
console.log('Error handling file ' + err);
})
.pipe(target);
// Attach new file to document list
target.on('close', function (file) {
filesUploadedList.push(file._id);
fileItem is obtained by iterating over the file req.files.files that download files.
However, I am trying to figure out the inMemory parameter for multer. If I set to true, then the buffer in fileItem will be filled. Just mark without this option the buffer / content of fileItem is empty, so in the code above it reads it from fileItem.location.
What's the best way to fill the target with content? fs.createReadStream makes the pipeline at the moment.
Thank.
J
source to share
You can accomplish this using a streamifier . The appearance of the code should look like this:
streamifier.createReadStream(fileItem.buffer).pipe(target);
Multer does not write anything to disk, now the target is filled with the buffer / contents of the loaded file.
source to share
You can use it like this:
(original idea here: stored files directly in mongodb using gridfs stream )
app.post('/upload',multer({
dest:"./tmp/",
upload:null,// take uploading process
inMemory:true, //or false, not needed here
onFileUploadStart:function(file){
//set upload with WritableStream
this.upload = gfs.createWriteStream({
filename:file.originalname,
mode:"w",
chunkSize:1024*4,
content_type:file.mimetype,
root:"fs",
metadata: {} // put some crazy meta data in here
});
},
onFileUploadData:function(file,data) {
//put the chunks into db
this.upload.write(data);
},
onFileUploadComplete:function(file) {
//end process
this.upload.end();
console.log('successfully written File to MongoDB Gridfs');
}
}),
function(req,res) {
res.sendStatus(200);
});
Since you are using the onFileUploadData () function, the Gridsfs will fill up as data arrives. This works regardless of whether inMemory is set to true or false. It should be noted that file.buffer [] can get very large if you are dealing with huge files. This is just a copy of all incoming data chunks.
In theory it would be better to use your own specific middleware on top of busboy without doing file operations (as used in Multer) at all.
source to share