Google DataStore with Lens - Composite Queries on HashMap

I am using Google App Engine for my REST service backend and Google DataStore as DB and objectify to access data store. One of the properties in my essence is

Map<String,String> customAttrs;

      

This is due to the fact that I do not know in front of me that all parameters can come from the client. My requirement is that I want to be able to index these parameters in a HashMap. I have considered the following question:

GAE w / Objectify - can you query the HashMap?

and tried what was mentioned in the accepted answer and it works for me, I can index based on attributes in customAttrs with

filter("customAttrs .key1", "value1")

      

based on any requests passed by the client in the REST API call.

However, the problem occurs with compound queries. I don't know much about this, but from what I understand, if you combine a sort or inequality filter with another filter, you must define the combination in a file named datastore-indexes.xml. So one of the fields in the entity is creationTime, and when I run any query that returns me a list of users, I want to sort it by creation time.

So let's say one of the keys on the CustomAttrs map is a city. Now if I just want to filter all users where they say city = London, it works great for me. However, if I try to get all users where city = London is sorted by createTime, it returns an exception and doesn't work. It only works if I add the following to the datastore-indexes.xml file:

<datastore-index kind="User" ancestor="false" source="manual">
        <property name="customAttrs.city" direction="asc" />
        <property name="creationTime" direction="asc" />
    </datastore-index>

      

This will now defeat the purpose of using a HashMap, since this is where I need to know the fields inside customAttrs before getting started.

So, is there a way to do compound queries without knowing the fields in front of your hand?

EDIT:

As Sai mentioned in answer, there is no way to accomplish this type of request without adding it before and adding it to datastrore-indexes.xml.

I was thinking of a different approach, but I ran into a different problem.

In this approach, instead of defining the property as

Map<String,String> customAttrs;

      

I have defined it as a list of strings

List<String> customAttrs;

      

Here, the strings that are stored are of the form key: value. For example, the list for an object could be "city: abc", "country: xyz", "name: 123"

In this case, I just need to define the following in my datastore-indexes.xml:

<datastore-index kind="User" ancestor="false" source="manual">
        <property name="customAttrs" direction="asc" />
        <property name="creationTime" direction="asc" />
    </datastore-index>

      

which is fine for me, since I don't hardcode / predefine what attributes can be anywhere. I can filter by city = abc using

filter("customAttrs","city:abc")

      

With this approach, I can filter by any of the parameters, and also sort the results using the createTime method.

But the problem with this approach is that I can only use one type of custom filter. Meaning, if I want to filter where city is abc and country is xyz (this works if I use Map approach), I cannot do that. Is there any way to do this. So basically my query is that if an object has a property that is List, is there a way to filter all those objects that have str1 and str2 in the List?

+1


source to share


2 answers


This has nothing to do with Objectfy, but the data store itself. Every time your query searches multiple properties, or sorts as you mentioned (searching and sorting on two different properties), the Datastore expects a corresponding composite index. The Datastore is designed so that all queries get results from a single index for performance. However, if you are allowing dynamic properties whose names are not known ahead of time, there is really no way to do the queries the way you want them to. If this is an option, your application should restrict the query / sort on these dynamic properties so that they do not require a composite index. Otherwise, you will have to research alternative options,such as full text search (AppEngine full text search or elastic search) and see if they help in your use case.



0


source


For your last question on filtering Array properties -

You can query AND

an Array query to see if it contains two or more items. GQL would like -



SELECT * FROM MyEntity WHERE ArrayProperty='value1' AND ArrayProperty='value2'

      

0


source







All Articles