Could this code cause excessive memory usage?
I am researching some code (written by someone else) to find out what is causing the memory usage to spike up to 8GB. I'm not a real hobby developer doing some coding at work, so it's a little over my head.
Could the following code, especially creating a new object for lin
each iteration, make the memory grow rapidly? There are about 3 million records being processed. The data in each column is nothing out of the ordinary.
conn
is a custom class.
SqlDataReader rdr = comm.ExecuteReader();
object[] lin = new object[17];
while (rdr.Read()){
rdr.GetValues(lin);
conn.Submit(lin, ref cmdString, ref recCount);
lin = new object[17];
}
I have seen some answers to memory related questions about the garbage collector that cannot keep up with the times. It seems to me that creating a new object at each iteration is not required.
source to share
It depends on how much memory is available. Since the last line of your loop body is overwriting lin
, no one seems to be referencing it anymore. Only suspects would be rdr
and conn
, since they may contain one for no apparent reason. If they don't, it will be available for garbage collection, which will take care of freeing up the resources of the arrays you initialized when invisible.
However, a cleaner solution might look like this, where the array gets cleaned up and reused instead of initializing a new one and overwriting the old variable that was used to store the previous one.
var lin = new object[17];
while (rdr.Read()){
rdr.GetValues(lin);
conn.Submit(lin, ref cmdString, ref recCount);
Array.Clear(lin, 0, 17);
}
Alternatively, you can move the array declaration to the inner scope, as I said in my comment:
while (rdr.Read()){
var lin = new object[17];
rdr.GetValues(lin);
conn.Submit(lin, ref cmdString, ref recCount);
}
I'm not sure which of these solutions will be faster, but I'm sure you can figure it out through trial and error.
source to share
In more detail on what Oppassum said, I think your "line" variable might be the culprit.
By doing a new operation on lin in every loop, you are allocating that memory and not reusing it. You are correct to create an object outside of the loop, as this reuses the memory of the object. I can't remember if rdr.GetValues () requires the array of objects you are passing through empty or not. It can just overwrite existing values, so cleaning up becomes unnecessary.
SqlDataReader rdr = comm.ExecuteReader();
object[] lin = new object[17];
while (rdr.Read())
{
rdr.GetValues(lin);
conn.Submit(lin, ref cmdString, ref recCount);
// Clearing, but may not be nessessary.
for(int i; i<lin.Length;i++)
lin[i] = null;
}
With all that the best way to figure out memory issues is still to use a profiler like RedGate.It comes with a 14-day free trial so you can find out if it helps you before buying it.
source to share