LOAD DATA INFILE + progress tracking
I created a PHP script console that downloads a large CSV file (> 2.5m lines) and runs it into the database. I am using LOAD DATA INFILE
query here and it works great, takes ~ 20 seconds.
I want to track the progress of this request. I've read about several approaches here, but I don't know how to start a request LOAD DATA INFILE
and then start a progress tracking loop. The script is awaiting execution LOAD DATA INFILE
.
My database is MySQL and the engine is InnoDB. I am using Laravel framework.
+3
source to share
1 answer
You can "split" the pt-fifo-split dump file and print a progress message after each chunk.
For example, in a shell:
f=dump.sql
nlines=`cat $f | wc -l`
let chunk=$nlines/100
pt-fifo-split --lines $chunk $f
i=0
while [ -e /tmp/pt-fifo-split ]
do
echo "$i% is done"
mysql -e "LOAD DATA INFILE '/tmp/pt-fifo-split' INTO ..."
let i=$i+1
done
0
source to share