How can I measure the overall memory requirements of a Python program
I have a financial pricing application written in Python 2.4.4 that works like an Excel plugin. Excel has a 1GB limit for all adds, so if any add process tries to allocate more than 1GB, Excel will crash.
I recently made a program change that may have changed the overall memory requirements of the program. I would like to work it out if something sigifnicly changed, and if not, then I can reassure my guide that there is no chance of failure due to increased memory usage.
By the way, you can run the same function that runs in Excel from the command line:
In fact, I can provide all the arguments that would be generated from Excel from a regular Windows prompt. This means that if I have a method to detect the memory requirements of a process on it, I can safely assume that it will use a similar amount when run from Excel.
I'm not interested in the details a memory profiler can give: I don't need to know how much memory each function is trying to allocate. What I need to know is the smallest amount of memory the program will need to run, and I need to ensure that if the program runs within 1GB, it will work fine.
Any suggestions as to how I can accomplish this?
The platform is Windows XP 32bit, python 2.4.4
source to share
I don't believe XP is tracking the peak memory requirements of a process (as Linux does in / proc / pid / status). You can use third party utilities like this one and set it up to "poll" the process very often to get a good chance of capturing the correct peak.
The best approach, although it doesn't answer your title question, is to try to start a process under the job object using a SetInformationJobObject with a structure JOBOBJECT_EXTENDED_LIMIT_INFORMATION
that has a suitable structure ProcessMemoryLimit
(for example, virtual memory with a maximum capacity of 1 GB) and a right flag in JOBOBJECT_BASIC_LIMIT_INFORMATION
to activate this limit. This way, if the process runs correctly, you KNOW that it has never used more than 1GB of virtual memory - otherwise it will crash with an out-of-memory error.
This programmatic approach also allows you to measure peak values, but it is quite complex compared to using the already packaged instruments that I do not think it can be recommended for this purpose. I am not aware of packaged tools that allow you to use job objects to specifically limit the resources available to processes (although I would not be surprised if you hear some of them available, whether free or commercially, I have never come across one myself) ...
source to share
The simplest solution is to simply launch the application from the command line and use the task manager to see how much memory it is using.
Edit: For something a little more complex take a look at this question / answer How to get memory usage under Windows in C ++
Edit: another option. Get Process Explorer . It is basically the task manager on steroids. It will record the peak usage of the process for you. (free so you don't need to worry about cost)
source to share