ASP.NET - Single Large Web Request Triggers System.OutOfMemoryException - There is still a lot of available memory
Environment: Windows 2003 Server (32 bit); IIS6, ASP.NET 2.0 (3.5); 4Gb Ram; 1 Workflow
We have a situation where we have a very large System.XmlDocument file loaded into memory and then passed into a subordinate XSL transformation. What happens when a web request arrives at the server is that it sits idle with 2500MB of available system memory.
When the XML DOM fills up, the available memory drops by about 500MB, at which point we receive the System.OutOfMemoryException event. At this point, the system should theoretically have 2000MB of available memory to service the request (as per Perfmon).
Related questions:
1) At what level in the stack is this constraint out of memory executed? OPERATING SYSTEMS? IIS? ASP.NET? the working process? Is this a limit on an individual web request?
2) Is this prefix configurable somewhere?
3) Why is this web request not allowing access to the full available system memory?
source to share
1) I would have guessed about a workflow, but that should be configurable in IIS up to the memory limit that a workflow can use. Another factor is what bit level your software uses, for example. 32 bits has a physical limit of 4 GB as this is the total address space.
2) Probably, but don't forget that memory fragmentation can play a role in getting out of memory faster than you think. if there is a memory request for a contiguous 1000MB chunk of memory, then it may not necessarily be in current memory.
3) Have you looked at the dump data to see what is in memory when the exception is thrown? If not, there are ways to take a snapshot of your memory to see what it looks like, as it might give you more information about what's going on.
source to share
- You are working in progress. A process can only access 2 gigabytes of memory. This task shares memory with everything else running in this process, so that bit of code doesn't get a full 2 gigs - even if it's available. There is also a 3-key switch on os. I believe this is a registry setting. But you'll have to search MSDN to find this information.
But realistically, you need to do it differently. Possibly by switching to a SAX style XML parser.
source to share
I'm sure there are some bright chapters here that might answer your specific questions, but did you ask if there is another way to do what you want? I'm specifically implying that you probably don't want to process a very large XML document, but you probably more specifically want to return something to the client. Could you rewrite the code to avoid this XML document altogether, or perhaps not load it all into memory at the same time and still give the same end result?
source to share
1) I don’t know. Check your magazines.
2) IIS limits memory mapped to websites / application pools. Check your settings.
3) Servers - all the time; if one application launches all the resources everyone else suffers. This is why enterprise applications like IIS limit memory to prevent escapes from taking over the entire server.
source to share