Clever Geek Handbook

Export CSV script out of memory

I have a PHP script that exports a MySQL database to CSV format. The strange CSV format is a requirement for a third party software application. It worked at first, but I now run out of memory on line 743. My hosting service has a 66MB limit.

My complete code is listed here. http://pastebin.com/fi049z4n

Are there alternatives to use array_splice

? Can anyone help me in reducing memory usage. What features can be changed? How do I free memory?

+3


source to share


2 answers


You must change your creation strategy CSV

. Instead, if you are reading the entire result from the database into an in-memory array, you should work in sequence:

  • Read line from database
  • Enter this line into a file CSV

    (or possibly a buffer)
  • Free the string from memory
  • Do it in a loop ...

This way, your script's memory consumption does not increase in proportion to the number of lines in the result, but remains (more or less) constant.



This can serve as an untested example to jot down the idea:

while (FALSE!==($row=mysql_fetch_result($res))) {
  fwrite ($csv_file."\n", implode(',', $row));
}
+7


source


Increase memory limit and maximum execution time

For example:



// Change the values to suit you
ini_set("memory_limit","15000M");
ini_set("max-execution_time", "5000");
+1


source






More articles:


All Articles