One big GridFS collection in MongoDB or much less?

We are using MongoDB to store video data before and after conversion. Encoding the results of one file from six or more smaller files (several grades of quality and different formats). Both input and output files have the same unique file identifier.

The question is which approach is better from a performance and scalability standpoint: store all output files in one huge gridFS collection with composite keys containing the original file ID, quality class and format, or have a separate collection for each quality / format pair?

For me both of these approaches have their pros and cons, i.e.

  • Using one large final collection uses less configuration at the time of reading, but a more complex query is required to find the file;
  • Using multiple result sets involves simpler and faster queries, but requires additional configuration to choose which collection this query should be processed into.

It's more like a matter of personal choice ... But what about scalability or performance? Has anyone come across such a solution before? Maybe someone can advise?

What is the best strategic approach given the large number of large files? Which approach will be easier to scale and delineate in the future? Maybe there are performance penalties in the long run?

+3
mongodb large-files gridfs blob


source to share


No one has answered this question yet

Check out similar questions:

1299
How can I query MongoDB with "how"?
1023
Text editor for opening large (giant, large, large) text files
873
Big data workflows using pandas
825
How to delete MongoDB database from command line?
4
Saving Images to GridFS MongoDB
4
Configuring GridFS Chunksize in MongoDB
3
Store folders in fs.files collection
3
Cleaning up lost files from GridFS
1
Mongodb Gridfs Queries - Metadata? Additional features?
0
Performance in MongoDB and GridFS



All Articles
Loading...
X
Show
Funny
Dev
Pics