EOutOfMemory Generating Large XML with Delphi
I am using Delphi to generate an XML document from data in a relational database. It does great with small datasets, but when I try to expand the dataset size to production levels it eventually throws an EOutOfMemory exception during node creation.
I am using TXMLDocument dropped in form (MSXML as provider) and my code looks something like this:
DS := GetDS(Conn, 'SELECT Fields. . . FROM Table WHERE InsuredID = ' +IntToStr(AInsuredID));
try
while not DS.Eof do
with ANode.AddChild('LE') do
begin
AddChild('LEProvider').Text := DS.FieldByName('LEProvider').AsString;
// Need to handle "other" here
AddChild('Date').Text := DateToXMLDate(DS.FieldByName('LEDate').AsDateTime);
AddChild('Pct50').Text := DS.FieldByName('50Percent').AsString;
AddChild('Pct80').Text := DS.FieldByName('80Percent').AsString;
AddChild('Actuarial').Text := DS.FieldByName('CompleteActuarial').AsString;
AddChild('Multiplier').Text := DS.FieldByName('MortalityMultiplier').AsString;
DS.Next;
end;
finally
DS.Free;
end;
with this section, as well as many other similarly constructed sections applied to various database tables, are executed many times. In this example, ANode is the IXMLNode passed to the function to use as a container.
I don't expect the resulting XML file to be more than 10 megabytes on disk. I am guessing that somehow I am leaking memory in my creation and disposal of XMLNodes, but I am not familiar enough with interfaces to know how to track down my problem.
source to share
TXMDocument is a DOM style interface and stores the entire document in memory. Memory used up so quickly. Even when the resulting file is not that big. You don't need TXMLDocument to write out simple XML. Why not write directly to an XML file?
That being said: This could also be a bug due to heap fragmentation or a real memory leak. You might want to try the tool mentioned here: Profiler and Memory Analysis Tools for Delphi
source to share
Each of these calls AddChild
has its own result, which is stored in a temporary variable IXmlNode
that is implicitly declared by the compiler. They should be cleaned up automatically when the current subroutine returns (usually as well as an exception). You can make your life more explicit by declaring your own variables.
var
le, child: IXmlNode;
begin
DS := GetDS(Conn, Format(Query, [AInsuredID]));
try
while not DS.Eof do begin
le := ANode.AddChild('LE');
child := le.AddChild('LEProvider');
child.Text := DS.FieldByName('LEProvider').AsString;
// Need to handle "other" here
child := le.AddChild('Date');
child.Text := DateToXMLDate(DS.FieldByName('LEDate').AsDateTime);
child := le.AddChild('Pct50');
child.Text := DS.FieldByName('50Percent').AsString;
child := le.AddChild('Pct80');
child.Text := DS.FieldByName('80Percent').AsString;
child := le.AddChild('Actuarial');
child.Text := DS.FieldByName('CompleteActuarial').AsString;
child := le.AddChild('Multiplier');
child.Text := DS.FieldByName('MortalityMultiplier').AsString;
DS.Next;
end;
finally
DS.Free;
end;
end;
The above code is missing implicit interface variables. The compiler had to declare a new implicit variable for each call AddNode
, but the code above demonstrates that only two are needed as it child
can be reused for every new child node.
However, this code should not cause excessive memory usage. Most likely, you are keeping references to objects you no longer need, or you are creating circular references for some of the interface objects. The MSXML library shouldn't create any circular references of its own, but you haven't listed all the code that can run here.
source to share