Clever Geek Handbook

Elasticsearch out of memory error

We are using elasticsearch 0.90.0 and java version "1.7.0_25". We transfer data from Oracle Oracle to Hadoop via an executable jar stored on a DB server. After 15-20 minutes of successful work, we get the following exception.

Caused by: java.lang.OutOfMemoryError: unable to create new native thread
    at java.lang.Thread.start0(Native Method)
    at java.lang.Thread.start(Thread.java:597)
    at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
    at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
    at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker.start(DeadLockProofWorker.java:38)
    at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:343)
    at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:95)
    at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:51)
    at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
    at org.elasticsearch.common.netty.channel.socket.nio.NioWorkerPool.createWorker(NioWorkerPool.java:45)
    at org.elasticsearch.common.netty.channel.socket.nio.NioWorkerPool.createWorker(NioWorkerPool.java:28)
    at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorkerPool.newWorker(AbstractNioWorkerPool.java:99)
    at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:69)
    at org.elasticsearch.common.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
    at org.elasticsearch.common.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
    at org.elasticsearch.transport.netty.NettyTransport.doStart(NettyTransport.java:240)
    at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:85)
    at org.elasticsearch.transport.TransportService.doStart(TransportService.java:90)
    at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:85)
    at org.elasticsearch.client.transport.TransportClient.<init>(TransportClient.java:179)
    at org.elasticsearch.client.transport.TransportClient.<init>(TransportClient.java:119)

No exception gets caught in either namenode / datanode or elasticsearch logs. This error hits the DB server, but I think it has something to do with looking for elasticity.

+3


source to share


1 answer


It looks like you are creating too many Netty clients, this is my guess, which in turn is consuming all your threads. Perhaps wrap your Netty client pool with the service that it introduces? See Helter Skelter's comment on this answer fooobar.com/questions/842587 / ...



0


source






More articles:


All Articles