Clever Geek Handbook

Loading data into Orderkey and VACUUM

I am uploading a night shot of a table to Redshift. I added a column called "rundate" at the end of the table, which is just displayed when the data has been fetched through my ETL process. It is also the main sorting key.

The tables keep getting longer and longer every night, and many have more than 400 columns.

I am currently using FILLRECORD along with EMPTYASNULL to get NULLS in the table, and once the COPY command has finished, I use

update table set rundate = 'date' where rundate is NULL

to have the correct snapshot date.

I am wondering if it still counts as "loading data in sort order" where I would not need to evacuate. In addition, no updates / deletions are performed on any of the records.

+3


source to share


1 answer


Unfortunately no. The update will do delete / insert, leaving all your load each day as dead records requiring VACUUM. I would recommend loading into an empty scripting table instead, and then inserting the data with an extra column after the session.



+3


source






More articles:


All Articles