Output buffering and large MySQL databases in PHP 5
I am trying to create an XML feed from a database with a ginormous table, almost 4k records. I want to use output buffering to make it spit out XML, but the script still keeps crashing.
ob_start();
$what = 'j.*, (select description from tb_job_type as jt WHERE jt.jobtype_id = j.job_type_id) as job_type,';
$what .= '(select description from tb_location as l WHERE l.location_id = j.location_id) as location,';
$what .= '(select description from tb_industry as i WHERE i.industry_id = j.industry_id) as industry';
$where = ('' != $SelectedType) ? 'j.job_ad_type="' . $SelectedType .'"' : '';
$process = $db->executeQuery('SELECT ' . $what . ' FROM tb_job_ad as j' . $where);
while($result = mysql_fetch_array($process))
{
$result['job_title_url'] = $form->urlString($result['job_title']);
$result['job_title'] = htmlspecialchars($result['job_title'], ENT_QUOTES, 'UTF-8');
$result['short_description'] = htmlspecialchars($result['short_description'], ENT_QUOTES, 'UTF-8');
$result['full_description'] = htmlspecialchars($result['full_description'], ENT_QUOTES, 'UTF-8');
$result['company_name'] = ucwords(strtolower($result['company_name']));
$tpl->assignToBlock('ITEMS', $result);
}
$cheese = ob_get_contents();
$actualize = $tpl->actualize('FEED');
header("Expires: Mon, 26 Jul 1997 05:00:00 GMT" );
header("Last-Modified: " . gmdate( "D, d M Y H:i:s" ) . "GMT" );
header("Cache-Control: no-cache, must-revalidate" );
header("Pragma: no-cache" );
header("Content-type: text/xml");
echo $actualize;
ob_flush();
print $cheese;
ob_end_clean();
This seems to be the line that does the choke script:
$tpl->assignToBlock('ITEMS', $result);
Help me please?
thank
Midiane.
source to share
The timeout is almost certainly happening because your query is slow - and you are almost certainly improving its performance by making sure you have null columns indexed and doing some JOINs.
What if you rebuilt your query build like this:
$q = 'SELECT
j.*,
jt.description as job_type,
l.description as location,
i.description as industry
FROM tb_jobs AS j
INNER JOIN tb_job_type AS jt ON j.job_type_id = jt.jobtype_id
INNER JOIN tb_location AS l ON j.location_id = l.location_id
INNER JOIN tb_industry AS i ON j.indsutry_id = i.industry_id';
if (! empty($SelectedType)){
$where = "WHERE j.job_ad_type = '$SelectedType'";
}else{
$where = '';
}
$q .= $where;
Make sure all your foreign key columns (j.location_id etc.) are indexed.
If you want the output to start earlier, you will want to - Query the database - List all your headers, etc. - write in a loop:
ob_end_flush();flush();
while($row = mysql_fetch_assoc($process)
{
ob_start();
//build your ITEM and echo it here
ob_end_flush(); flush();
}
PS: I also noticed another SQL related question about you here on SO. I think it will serve you very well to understand better how SQL works. I highly recommend getting a copy of " " SLQ Clearly Explained "- it exactly matches the promises header - explicit SQL explanation (in general, no linking discussion of different implementations)
source to share