How to connect Google bigtable outside of Google cloud platform

I have a Google Bigtable cluster created. I am trying to connect the following this tutorial I tried to load my cli project and build it but when I try to execute any command it fails. Also I tried to build my simpler connector example. Here he is:

import org.apache.hadoop.hbase.client.Admin;
import org.apache.hadoop.hbase.client.Connection;
import org.apache.hadoop.hbase.client.ConnectionFactory;
import org.apache.log4j.Logger;

import java.io.IOException;
import java.util.ArrayList;

public class BigTableTestConnector {
    private final static Logger logger = Logger.getLogger(BigTableTestConnector.class);


public void testConnection() throws MasterNotRunningException, ZooKeeperConnectionException {

    try {
        Connection connection  = ConnectionFactory.createConnection();

        String tableName = "testTable";
        ArrayList<String> columnFamilies = new ArrayList<String>();
        columnFamilies.add("columnFamily1");
        columnFamilies.add("columnFamily2");
        columnFamilies.add("justString");
        columnFamilies.add("uhhaha");

        Admin admin = connection.getAdmin();
        HTableDescriptor tableDescriptor = new HTableDescriptor(tableName);
        for (String colFamily : columnFamilies) {
            tableDescriptor.addFamily(new HColumnDescriptor(colFamily));
        }
        admin.createTable(tableDescriptor);

    } catch (IOException e) {
        e.printStackTrace();
    }


    logger.info("done");
}

public static void main(String... args) throws Exception {
        BigTableTestConnector bttc = new BigTableTestConnector();
        bttc.testConnection();
    }
}

      

but when i run it on my local machine i get the following exception

java.io.IOException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:240)
at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218)
at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:90)
at com.zoomdata.thrift.provider.BigTableTestConnector.testConnection(BigTableTestConnector.java:33)
at com.zoomdata.thrift.provider.BigTableTestConnector.main(BigTableTestConnector.java:81)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)
... 9 more
Caused by: java.io.IOException: Error getting access token from metadata server at: http://metadata/computeMetadata/v1/instance/service-accounts/default/token
at com.google.cloud.bigtable.hbase.CredentialFactory.getCredentialFromMetadataServiceAccount(CredentialFactory.java:100)
at com.google.cloud.bigtable.hbase.BigtableOptionsFactory.fromConfiguration(BigtableOptionsFactory.java:236)
at org.apache.hadoop.hbase.client.BigtableConnection.<init>(BigtableConnection.java:120)
... 14 more
Caused by: java.io.IOException: ComputeEngineCredentials cannot find the metadata server. This is likely because code is not running on Google Compute Engine.
at com.google.auth.oauth2.ComputeEngineCredentials.refreshAccessToken(ComputeEngineCredentials.java:63)
at com.google.auth.oauth2.OAuth2Credentials.refresh(OAuth2Credentials.java:76)
at com.google.cloud.bigtable.hbase.CredentialFactory.getCredentialFromMetadataServiceAccount(CredentialFactory.java:98)
... 16 more
Caused by: java.net.UnknownHostException: metadata
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
at sun.net.www.http.HttpClient.New(HttpClient.java:308)
at sun.net.www.http.HttpClient.New(HttpClient.java:326)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1168)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1104)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:998)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:932)
at com.google.bigtable.repackaged.com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:93)
at com.google.bigtable.repackaged.com.google.api.client.http.HttpRequest.execute(HttpRequest.java:965)
at com.google.auth.oauth2.ComputeEngineCredentials.refreshAccessToken(ComputeEngineCredentials.java:61)
... 18 more

      

He said:

ComputeEngineCredentials cannot find the metadata server. This is probably due to the code not working in Google Compute Engine.

Does this mean that this code can only work inside the Compute Engine? Is there a way to connect to it from my local computer? Does anyone have any examples of this?

+3


source to share


1 answer


To run locally, you need to make sure the environment variable is GOOGLE_APPLICATION_CREDENTIALS

set to yours key.json

obtained from the cloud console.This is the likely cause in your case.



+3


source







All Articles