Using dynamic models in the Django framework

I am currently using the Django framework, including the model engine, to abstract the database schema declaration and database sharing, which is great for most scenarios.

However, my application also requires tables to be created and accessed dynamically at runtime, which as far as I can see is not supported by Django out of the box.
These tables usually have the same structure and can be largely abstracted by the same Model class, but Django does not allow you to modify the base db_table of a particular model query, since it is declared in the Model class, not the Manager.

My solution for this is to do this process whenever I need a new table to be created, populated and available:

• Create and populate table using raw sql
• Add indexes to table using raw sql

• When I need to access a table (using the django queryset api), I declare a new type dynamically and return it as a model for the query using this code:

  table_name = # name of the table created by sql
  model_name = '%d_%s' % (connection.tenant.id, table_name)
  try:
      model = apps.get_registered_model('myapp', model_name)
      return model
  except LookupError:
      pass
  
  logger.debug("no model exists for model %s, creating one" % model_name)
  class Meta:
      db_table = table_name
      managed = False
  
  attrs = {
     'field1' : models.CharField(max_length=200),
     'field2' : models.CharField(max_length=200),
     'field3' : models.CharField(max_length=200)
     '__module__': 'myapp.models',
     'Meta':Meta
  }
  
  model = type(str(model_name), (models.Model,), attrs)
  return model
  
        
  
          
          
          
        
  
      

• Note that I am checking to see if the model has already been registered with django and I am using the existing model in case this happens. The model name is always unique for each table. Since I am using multiple tenants, the tenant name is also part of the model name to avoid conflicts with similar tables declared in different schemas.

• If this is not clear: tables created dynamically will and should be persistently persisted for future sessions.

This solution works great for me so far. However, the application will need to support a large number of these tables. those. 10,000 - 100,000 such tables (and corresponding model classes) with up to a million rows per table. Assuming the underlying db is ok with this load, my questions are:

• Do you see any issues with this solution, with and without the expected scale?

• Does anyone have a better solution for this scenario?

Thank.