QueryIncompleteError: Your request did not complete after 300 seconds. Most likely something is wrong on our side
When executing a pull request using the Python Keen client, we constantly encounter the same error:
Message: Your request did not complete after 300 seconds. Most likely, something is wrong on our side. Please let us know at team@keen.io.
Code: QueryIncompleteError
Request parameters: (in JSON format)
{
"timezone": -18000,
"event_collection": "Loaded a Page",
"filters": [
{
"operator": "eq",
"property_name": "reportType",
"property_value": "Profile"
}
],
"timeframe": {
"start": "2017-04-24",
"end": "2017-06-19"
}
}
My suspicions are that the requested date range is too large and the Keen API is choking on the size of this dataset, but it's not clear from the error message.
source to share
Your guess is correct! This 504 error occurs when your request expires (takes more than 5 minutes). The following are ways to shorten the execution time of your request:
1. Shorten the timeframe in your request
The smaller the timeframe, the faster the request. A request for one week of data will be 4x faster than a request for 1 month of data (approximately).
A relatively easy fix would be to split this query into two queries, dividing the timeframe into two or more parts.
2. Reduce the number of extracted properties.
The extraction request type takes a parameter property_names
. Here you can specify the array of properties you want when retrieving. Without this parameter, the API will return all the properties of your events. Using property_names
only the properties you need to retrieve can significantly reduce the computational cost and overhead for the query.
3. Reduce the size of your collection
This probably doesn't apply in this case because it looks like your data model is already set and sane, but a query parsing 400M events would take about twice as long as a query parsing 200M events. For this reason, we do not recommend storing all types of events in one mega collection. Keen was designed to have multiple collections for each type of action (registration, discovery, messages, etc.). For example, if you narrowed this collection down on the Loading Profile Page, your query would be much faster as it would not need to sort all other report types.
For others experiencing this error, caching can also reduce request response times to milliseconds. Caching works for all types of analysis like count, sum, median, etc., except for extractions.
source to share