How to store files in Azure
We need to store ~ 50k files every year. Each file is 0.1-5MB in size, which corresponds to the 5GB - 250GB range. Files: jpg, avi, pdf, docx, etc.
We were just storing BLOB files in sql server, but I think this is not the best idea in this scenario because the database is going to be huge in 2 years.
What would be the best way to store this data?
I see a lot of different options and can't figure out where to start: Azure, Azure SQL, etc. Newer versions of SQL Server also have several hybrid versions.
source to share
I am using the following approach for multiple systems.
- Azure storage for files. You can create multiple containers and access levels if they store sensitive information.
- Azure CDN to serve static content from Azure Storage
- Azure DB as a database engine
In Azure DB, I store the file path, with some additional processing in my applications, how to access the file to generate the final url to serve the file. This is because the CDN does not support SSL on custom domains.
If you need examples or more information, just let me know. I am sitting at the airport, so I provide a slightly less detailed answer.
source to share
As @Martin pointed out in his answer in his answer, Azure Storage is viable, in part because:
- Access is independent of any virtual machine, service, database, etc.
- Storage is durable - with triple replication within an area and possibly geo-replication
- Individual droplets can be up to 200GB, with storage scaling up to 500TB
Azure also provides a file service, which is essentially an SMB site sitting on top of blobs.
Although Azure has database services (SQL Database Service and DocumentDB), you will find that they are not optimized for BIL; they are more optimized for metadata. You can store binary data in each of them, but you need to worry about storage limits.
You can also deploy your own database solution through virtual machines along with attached disks (again backed up by reliable memory storage). Virtual machines support up to 32 1TB drives attached to a given virtual machine, whether they are regular blobs or "premium" SSB keyfobs (each premium drive supporting up to 5000 IOPS). Which database solution you choose is entirely up to you; there is no "best" solution.
source to share