How do I access my DataStore AppEngine objects from my Compute Engine VM?
My application is running in App Engine, but I would like to access its NDB DataStore objects from my VM machine in order to do some processing and write the results back to the AppStore DataStore. How can i do this?
Also, are Google DataStore and App Engine DataStore data the same? https://developers.google.com/datastore/ https://developers.google.com/appengine/docs/python/ndb/
source to share
David's solution requires using the App Engine instance time to execute queries, but you can work around it and access the Datastore directly from the Compute Engine instance. There is a pretty good tutorial on how to do this. But it's not as pretty as ndb.
>>> import googledatastore as datastore
>>> datastore.set_options(dataset='project-id')
>>> req = datastore.BeginTransactionRequest()
>>> datastore.begin_transaction(req)
<datastore.datastore_v1_pb2.BeginTransactionResponse object at ...>
source to share
Google Cloud Datastore is a kind of standalone version of the App Engine datastore.
Coming back to your problem, you have two options:
-
Write a web service to expose your entities from an App Engine application to Compute Engine VMs. One option: Cloud endpoints . Cloud endpoints use OAuth2 authentication and Compute Engine VMs come with OAuth2 accounts that you can use to authenticate the service .
-
Use the App Engine API to access the datastore. The remote API provides access to the data store as if you were on an App Engine instance. But by default the API requires you to provide a password for the App Engine amdin, so you might have to store your password in Compute Engine VMs, which is insecure.
source to share
Besides the options that @David explained, you can also look at managed VMs : they are the crossroads between Compute Engine and App Engine - basically a managed Compute Engine instance that has access to App Engine services.
source to share
The officially recommended way is to use the data warehouse client libraries; see https://cloud.google.com/datastore/docs/reference/libraries You need to create a service account or use the default computing machine service account, grant permission to all APIs or data store for this service account and create an instance of a computing engine that will be part of this service account. See here for details . Then you can do something like:
from google.auth import compute_engine
from google.cloud import datastore
datastore_client = datastore.Client(project='yourproject',
credentials=compute_engine.Credentials())
q = datastore_client.query(kind='YourEntity')
q.add_filter('field_name', '=', 'HelloThere')
print list(q.fetch(1))
source to share