Memory profiling of a multithreaded haskell program

I have a Snap web app that serves some JS files and 1pixel images (the main task is to run rather quickly and then serve huge html media content). There are several servers behind HAProxy.

I updated it from GHC 7.6 to 7.8 and also updated some libraries. After the update, the application gradually started flowing (on all servers), ending in OOM every 15 minutes on 8GB RAM machines (and much longer by 16GB) and restarting afterwards.

The problem is that if I compile the app for a while to profile and run the app, I no longer see any memory leaks. It just consumes 1 processor and runs in persistent small memory.

So, I wanted to ask for general advice on how to find such a bottleneck if working under profiling doesn't help.

UPDATE: After playing -A100M

around with the app, I noticed that if I remove the runtime parameter it won't OOM, which is fast, but with the default HAProxy "sessions" get to the limit (so it mostly chokes). I am currently playing around with various RTS flavors, hopefully some of them will help me get both performance and long term memory consumption.

UPDATE 2: Just for the record, I found that when using the -A30

rts of the optional app, being hungry, lives quite well. 8Gb devices OOM-kill, but 16Gb looks like this: http://i.imgur.com/3W9KpFS.png (green line "RAM available", the procedure that restarted the application in the graph). I'm happy with the result, but would love to know any methods for profile memory of a multithreaded application anyway.

UPDATE 3: I vote to close this question as "too broad". In general, I see that if such a generic set of tools that allow you to simplify simple profile memory exist, they will definitely be documented elsewhere on the wiki, etc.

+3


source to share


1 answer


I've never used it myself, but maybe tick-tick profiling could help? It should be immune to optimization changes caused by normal profiling, but at the cost of a more complex interpretation.



Basically compile and link appropriate modules with flags -ticky

and -rtsopts

and run with flag +RTS -rfoo.ticky

to get heaps of data into foo.ticky

.

+2


source







All Articles