ElasticSearch Multiple data directories, choose where to place the index
I am running ElasticSearch across multiple servers. All servers are equal, all have 2 drives: one SSD and one hard drive. Needless to say, SSDs are faster and smaller.
I know that you can install multiple data directories in ES by adding paths to elasticsearch.yml
. But by default (from what I found) ES automatically chooses which data directory to take based on the percentage of available disk space.
Some indexes are more important to me than others, saying that there should be newer ones on the SSD (those that require a lot of queries) and those that will be queried less might be on the hard drive.
What do I need to do to do this - if it is currently possible?
Add index to path?
source to share
This is not possible in Elasticsearch. Yes, you can specify multiple data paths, but they cannot be "assigned" to indices.
Currently ES will split data at the file level into all data paths, which means that shards will propagate on all paths:
The path to the directory where to store the index data allocated for this node.
path.data:/path/to/data p>
Multiple locations can be optionally included, resulting in the data being locations (a la RAID 0) at the file level, which favors a location with most free space to create. For example:
path.data:/path/to/data1,/path/to/data2
In 2.0, on the other hand, this will change , but you still cannot have an index on a data path and another index on a different data path.
Currently the only solution for your use is to have SSDs on hot nodes and hard drives on cold nodes, which means one ES node with SSD and one ES node with HDD: https://www.elastic.co/guide /en/elasticsearch/guide/current/retiring-data.html#migrate-indices
source to share