Clever Geek Handbook

Use elastic search with RabbitMQ

I have a RabbitMQ queue that is ingesting some documents that need to be migrated to Elastic Search.

How to create Elastic Search as a consumer so that ES acts on the user in the document queue.

Possible options:

  • Using RabbitMQ River?
  • Using RabbitMQ plugin? (How to do it)
  • Other?

Can anyone post an example?

+3


source to share


2 answers


on your ELK create config file /etc/logstash/conf.d/anyfile.conf



input {
   rabbitmq {
      host => 'rabbit.example.com'
      queue => 'my_queue_name'
      exchange => "my_exchange_name"
      key => 'my_logs'
      durable => true
   }
}
output {
   elasticsearch {
      host => "elk.example.com"
   }
}
+3


source


Since you are asking for RabbitMQ river , here is an example, although you should note that rivers will be deprecated soon (i.e. with ES 1.5) and Prameswar Lal's other solution using Logstash would be preferred.



curl -XPOST localhost:9200/_river/custom_river_name/_meta -d '{
    "type" : "rabbitmq",
    "rabbitmq" : {
        "host" : "localhost",
        "port" : 5672,
        "user" : "guest",
        "pass" : "guest",
        "vhost" : "/",
        "queue" : "elasticsearch",
        "exchange" : "elasticsearch",
        "routing_key" : "elasticsearch",
        "exchange_declare" : true,
        "exchange_type" : "direct",
        "exchange_durable" : true,
        "queue_declare" : true,
        "queue_bind" : true,
        "queue_durable" : true,
        "queue_auto_delete" : false,
        "heartbeat" : "30m",
        "qos_prefetch_size" : 0,
        "qos_prefetch_count" : 10,
        "nack_errors" : true
    },
    "index" : {
        "bulk_size" : 100,
        "bulk_timeout" : "10ms",
        "ordered" : false,
        "replication" : "default"
    }
}
0


source






More articles:


All Articles