Django Haystack - ElasticSearch result window is too large

I am working on a web application developed with Django that uses Postgres and ElasticSearch. It uses ElasticSearch as the database stores a lot of data that grows by the hour. To get things up and running quickly, I used django-haystack 2.6.0, which allows me to easily access Elastic Search. So far so good.

The problem came when I made a request to ElasticSearch and the result was really great. The error I received was the following:

TransportError: TransportError (500, u'search_phase_execution_exception), u'Result window is too large, from + size should be less than or equal to: [10000], but was [12602]. See scrolling api for a more efficient way to query large datasets. This limit can be set by modifying the index level parameter index.ax_result_window. ')

After some research, I tried to increase the size of the result windows, but this will not work in the long run as the queryset continues to grow in size. Also, django-haystack does not implement the scroll api, so I cannot follow the guidelines shown in the stack trace.

My question is, is there anyway I can fix this error and get a large set of queries with django-haystack, perhaps paginating the results in some way? Or is there a better approach without using a haystack?

Thanks in advance!

+3
python database django elasticsearch django-haystack


source to share


No one has answered this question yet

Check out similar questions:

677
Detecting peaks in a 2D array
53
Elasticearch 2.1: results window is too large (index.max_result_window)
12
Django EdgeNgramField sandwich gave different results than elasticsearch
five
Haystack and Elasticsearch: Limit Results
4
Django elasticsearch transport error request not found
2
django-haystack elasticsearch multiple indexes incorrect results
2
django haystack: how to iterate over all indexed elements
0
During Haystack update_index: "Result window is too large, size + should be less than or equal to: [10000], but was [10100]
0
How to query for empty MultiValueField results in Django Haystack
0
Results window is too large for elastics



All Articles
Loading...
X
Show
Funny
Dev
Pics