Best Upgrade Method for MySQL DB

I've read solutions for similar problems, but they all seem to contain scripts and additional tools. I hope my problem is simple enough to avoid this.

So the user uploads the csv data next week. It is inserted into the DB, no problem.

BUT

after an hour he gets feedback from everyone and should update accordingly. It updates the csv and submits it to the database.

Right now, on the system, I am using checks to see if there is data for this week, and if so, fetches all the data from the DB, the script finds the differences and sends them, and after all that, the old data is deleted and replaced with the new data.

Obviously it's much easier to just erase it and re-enter the data, but not the best method, especially if there are a lot of changes or tons of data. But I need to know THAT changes were made to send alerts. But I don't need the transaction log as alerts only need to be sent once and then the old data is useless.

So!

Is there a way to compare the new data with the already existing data, get only the rows that have been changed / deleted / added, and make those changes? Right now it seems like I can do an update, but then I won't get any response to what has changed ...

Thank!

Quick edit:

No foreign keys are used at this time. This will change soon, but it shouldn't make any difference, because foreign keys will only indicate who is affecting the data and therefore won't need to change. With regard to primary keys, this is a bit of a dilemma:

The data in question is the work schedule. So it would be nice (for the specific applications of this schedule other than simple output) for each shift to have a key. But the problem is, let's say that user1 is late for Monday. The delay is recorded in a separate table and linked to the offset using the toggle key. But if on Tuesday some changes need to be made for the past week, I fear that it will be too difficult to ensure that all DB entries that have already happened (and therefore may have associations that should not be broken) will be re-entered into action in the process. Unfortunately, this is not as easy as just updating all the events that occur AFTER the current time, as this will add work (and therefore make it less accessible) for people.which are loading. Basically, they schedule one program at a time, export it to CSV, and then upload it to a web page for all web applications that need this data. So it's just a lot easier for them (and less stressful for everyone involved) to do the same routine every time they export an entire week and load it.

So my biggest problem is to make the loading script as smart as possible from both ends. It doesn't get bloated trying to find changes, it can find changes regardless of input AND none of the data that doesn't change gets rekeyed.

Here is a related question:

Suppose Joe User was schedule to wash dishes from 7:00 PM to 8:00 PM, but the new
data has him working 6:45 PM to 8:30 PM.  Has the shift been changed? Or has the old
one been deleted and a new one added?

      

And one more:

Say Jane was schedule to work 1:00 PM to 3:00 PM, but now everyone has a mandatory
staff meeting at 2:00 to 3:00. Has she lost one shift and gained two? Or has one
shift changed and she gained one?

      

I'm really interested in knowing how such data is usually handled or approximated, rather than specific answers to the above.

Thanks again.

0


source to share


2 answers


Right now, on the system, I am using checks to see if there is data for this week, and if so, fetches all the data from the DB, the script finds the differences and sends them, and after all that, the old data is deleted and replaced with the new data.

So your script knows the differences, doesn't it? And you don't want to use additional additional tools besides your script and MySQL, right?



I'm pretty sure MySQL doesn't offer any "diff" tool by itself, so it's best to make a new CSV file just for updates. I mean - it should only contain the changed lines. The update will be faster and all modified data will be readily available.

0


source


If you have a unique key in one of the fields, you can use:



LOAD DATA LOCAL INFILE '/path/to/data.csv' REPLACE INTO TABLE table_name

      

0


source







All Articles