Can PermGen seep into its own heap?
We launch the Glassfish app with the settings below:
-XX:PermSize=1g
-XX:MaxPermSize=2g
-Xms4g
-Xmx4g
-XX:MaxDirectMemorySize=1048576
JDK version - 6u45
We are seeing a memory leak - the Java Glassfish process is constantly growing - RES
getting> 15GB (our server has 16GB of physical memory, so admins restart Glassfish before they hit physical limits), but we never get java.lang.OutOfMemoryError
.
Output jmap -permstat
after running the application for a long time showing that the classloaders are taking up ~ 9.5 GB (!!!) :
class_loader classes bytes parent_loader alive? type
<bootstrap> 4242 24064120 null live <internal>
0x0000000794a4c030 20 274072 0x0000000794a4c098 dead groovy/lang/GroovyClassLoader$InnerLoader@0x0000000609cdc9f0
...
<many, many Groovy class loaders>
0x000000077f9c80d8 0 0 0x00000007017859f8 dead groovy/lang/GroovyClassLoader@0x0000000609997f00
0x000000076d63b3e0 0 0 0x00000007017859f8 dead groovy/lang/GroovyClassLoader@0x0000000609997f00
0x000000075a4c5248 20 261784 0x000000075a4c52b0 dead groovy/lang/GroovyClassLoader$InnerLoader@0x0000000609cdc9f0
0x000000078ea2e998 0 0 0x00000007017859f8 dead groovy/lang/GroovyClassLoader@0x0000000609997f00
...
total = 73518 745295 9690318280 N/A alive=1, dead=73517 N/A
At the same time jmap -heap
shows PermGen using 700MB:
PS Perm Generation
capacity = 2147483648 (2048.0MB)
used = 823831536 (785.6669769287109MB)
free = 1323652112 (1262.333023071289MB)
38.362645357847214% used
I always thought that with help -XX:MaxPermSize
and -Xmx
you can control how much memory the java process will allocate (throwing OOM more if necessary). I understand that this process should roughly match:
4G (heap) + 2G (PermGen) + N*1MB (N thread stacks) ~= 7.5G (by 500 threads)
Question
We know the leak is coming from an old version of Groovy and missing PermGen flags, but how is it possible that classloaders are taking up 9.5GB of memory that does not belong to either PermGen or the java heap?
Edit
Just a clarification - the question is not how to solve the memory leak problem, but why the JVM allows such memory allocation .
source to share
Not much of an answer, but whatever. It's complicated. Keep in mind that the JVM is going to use extra wiggle room for Garbage collection
(especially G1), Metaspace: Permgen in Java8 (which doesn't apply in your case), JIT optimizations, and direct buffers that might be outside the usual trash can.
Either way, analyze the heap dump to see what the application behavior is. Generally the error is present in the application than in the JVM. Also you have to let the application run in OutOfMemory
and see exactly what reason is mentioned.
source to share