ElasticSearch Crashing repeatedly "HeapDumpOnOutOfMemoryError" despite 4GB ES_MAX_MEM
I am running ElasticSearch via
echo "export ES_HEAP_SIZE = 4096" >> /root/setenv
echo "export ES_MAX_MEM = 4096" >> /root/setenv
echo "export ES_MAX_MEM = 4096" >> /root/setenv
# finally, we can start the app
echo 'Starting ElasticSearch...'
bin/elasticsearch -Xmx4g -Xms4g
However, after a while (20 minutes), he becomes unresponsive, apparently due to HeapDumpOnOutOfMemoryError
...
[root@ip-***** api]# ps -ax | grep elasticsearch
6225 ? SLl 5:35 java -Xms256m -Xmx1g -Xss256k -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -Delasticsearch -Des.path.home=/root/elasticsearch/elasticsearch-0.20.3 -cp :/root/elasticsearch/elasticsearch-0.20.3/lib/elasticsearch-0.20.3.jar:/root/elasticsearch/elasticsearch-0.20.3/lib/*:/root/elasticsearch/elasticsearch-0.20.3/lib/sigar/* -Xmx4g -Xms4g org.elasticsearch.bootstrap.ElasticSearch
EDIT - I just noticed that the -Xms size for output here is only 256m despite running 4g at start above. I do not understand something?
FWIW, I'm on Amazon EC2 (m1.large instances => 8GB RAM) running CentOS and Java v1.6.0_14-b08
source to share
First, installation ES_HEAP_SIZE
will not help you prevent ouf-of-memory errors. The amount of memory used by Elasticsearch is dictated by the type of queries you make: whether you are using cutting, sorting, filtering, how many fields, how large those fields are, what their cardinality is, etc.
Second, it is preferable to use a ES_INCLUDE
script, or a service wrapper , rather than passing all the parameters you need on the command line to the elasticsearch
script.
Third, if you set your environment variable correctly ES_HEAP_SIZE
, you don't need to pass parameters -X
to the elasticsearch script. In fact, these parameters have no effect - the script will not pass them to Java. Use a variable ES_HEAP_SIZE
to manage memory, and use ES_JAVA_OPTIONS
to manage additional variables that you would like to pass to Java.
source to share