GPU Tensor on Google Cloud
I have a model for Google Machine Learning using tensorflow and everything is fine. Now I want to make some predictions with the GPU. I saw this link, but it talks about GPU training, not prediction. There is nothing about the GPU in the prediction session.
Does anyone know if it is possible to predict it with a google machine learning mechanism with a GPU? Or if I use GPU training, is my prediction automatically triggered from the GPU?
I am using the following command line:
gcloud ml-engine predict --model ${MODEL_NAME} --json-instances request.json
This command works, but it uses the CPU. Additional information: My model is published in the us-east1 zone and my scale is automatic.
source to share
You cannot use GPU for prediction in ml-engine. It's unclear if they are using the default GPUs - I would link to the documentation, but there are no available ones.
I am sure, however, that they do not use TPU. Google currently only uses TPU for internal services; although they created a TPU cloud exclusively for researchers to experiment with: https://yourstory.com/2017/05/google-cloud-tpus-machine-learning-models-training/
If you need more control over how your prediction is performed, for the same price, you can tune the Google Compute Engine with the powerful Tesla K80 GPU. Your Tensorflow model will work too and is easy to customize.
My suggestion would be to make predictions using your GCE instance and then compare them against the ml-engine. If the ml-engine is faster than GCE, Google is probably using GPUs for prediction. Surely their goal is to provide GPU and TPU as the ml engine in the future, but demand is overwhelming the HPC cloud today.
source to share
This site has information:
https://cloud.google.com/ml-engine/docs/how-tos/getting-started-training-prediction
However, this is a completely different way of teaching and predicting. They provide a means to train and predict their service infrastructure. You just build the model with your Tensorflow program and then use your hardware with the cloud SDK. So there is no need to worry about whether it runs on CPU or GPU.
source to share