Large Scale Business Application - Cloud Realm and Azure Storage Issues
We are building a large scale database driven business application. This business application is multi-tenant and we want to provide it as SOA (Software as a Service) in the Azure cloud. Azure SQL Database is limited to 150 GB, which is not enough for us. Our clients have a huge amount of data that is likely to increase every year. Our application database is relational and complex and uses complex queries (Groups, Joins, Aggregate Functions), stored procedures to retrieve / save data. Therefore, we are reluctant to use Big Table directly. There is a suggestion to split the database into partitions and filegroups, but we are not going to use it.There is another possibility to save only the current year data in the application and rest in another partition / backup database, but this is not suitable for our application as it is often accessed by past data. If we use an Azure Table, our complex operations will be performed in memory, which is not possible due to limited memory resources. We are thinking, however, to create a combination of Azure SQL Database and Azure Table. But we cannot find a suitable example that uses a combination of both.But we cannot find a suitable example that uses a combination of both.But we cannot find a suitable example that uses a combination of both.
We are also looking for answers to the following questions:
- Is azure suitable for storing very large relational data (TB) or do we have our own database servers to support this?
- Should we only use Azure Table and NOT Azure SQL?
- Should we use a mix of Azure SQL and Azure Table storage? Is there a sample architecture / example to guide us on this?
source to share
The new Standard and Premium Azure SQL Database SKUs now support 250GB and 500GB respectively. Regardless, Azure SQL Database is a very large scale model for large amounts of data. A preview of the resilient scale feature for Azure SQL Database has just been announced .
There are good reasons for storing data in the most appropriate place - Azure Tables provides much more cost-effective and high-performance storage than Azure SQL Database. There are specific reasons for using each of the best suited data — high-level, semi-structured data in Azure tables and relational data in Azure SQL Database.
1) If you are creating a new application, you should consider Azure SQL Database for your relational data and only switch to SQL Server in Azure Virtual Machines if that doesn't work. While SQL Server in a VM seems lightweight, things start to get more complex when you start adding traditional DBA activities like HA deployment. If you have high-level relational needs, you should consider sharding.
2) Your choices should not be one or the other, but about using them where appropriate.
3) Microsoft Patterns and Practices has a number of e-books that you should look out for to help you with your architecture.
source to share