Postgresql Table Partitioning Django Project

I have a Django 1.7 project that uses Postgres 9.3. I have a table that will be quite large. The table will have between 13 million and 40 million new rows per month.

I would like to know what is the best way to enable Postgres table splitting in Django?

+3


source to share


3 answers


As long as you use inheritance and then hook up the parent table to your Django model, sections should be completely transparent to Django. That is, SELECT

the parent table will cascade down to partitions unless the keyword is explicitly used ONLY

(where applicable).

Note that separation makes it difficult to implement a programmatic method for determining when to create new sections and then create them - or do it manually at regular intervals. Depending on your exact data and business logic, it is likely that you might also need to implement triggers and rules to determine which section is, say, INSERT

something (since you would not want to be INSERT

in the parent table). However, they should also be abstracted from Django.



I found that depending on the specific circumstances, this might be required when shutting down the main application, as creating a new section causes a deadlock.

It is also worth considering whether you need true partitions that are created over time, or if the inheritance model is sufficient, such as tables foo

and foo_archive

, where foo_archive

inherits from foo

and periodically something (such as a script) moves old data in foo_archive

to shrink foo

.

+5


source


You can use Architect app for Postgresql Table Partitioning Django Project

Architect's implementation of PostgreSQL partitioning is done purely at the database level. This means that the Architect creates multiple triggers and functions and inserts them directly into the database, so even if you issue a direct insert statement from the database console and not from the ORM, everything will work as expected and the record will be inserted into the correct section if the section does not exist, it will be created automatically for you. Sections can also be created in any order, not just lower to higher.



This is a new version of the old Django DB Parti app

+9


source


If you are using a newer PostgreSQL version you can try this

https://github.com/chaitin/django-pg-timepart

A Django extension that implements PostgreSQL tables for partitioning and date-based management.

0


source







All Articles