Load data into table
I'm trying to set up Hive to work with JDBC and I used this example for eclipse:
public class HiveJdbcClient {
private static String driverName = "org.apache.hadoop.hive.jdbc.HiveDriver";
/**
* @param args
* @throws SQLException
**/
public static void main(String[] args) throws SQLException {
try {
Class.forName(driverName);
} catch (ClassNotFoundException e){
// TODO Auto-generated catch block
e.printStackTrace();
System.exit(1);
}
Connection con = DriverManager.getConnection("jdbc:hive://localhost:10000/default", "", "");
Statement stmt = con.createStatement();
String tableName = "testHiveDriverTable";
stmt.executeQuery("drop table " + tableName);
ResultSet res = stmt.executeQuery("create table " + tableName + " (key int, value string)");
// show tables
String sql = "show tables '" + tableName + "'";
System.out.println("Running: " + sql);
res = stmt.executeQuery(sql);
if (res.next()) {
System.out.println(res.getString(1));
}
// describe table
sql = "describe " + tableName;
System.out.println("Running: " + sql);
res = stmt.executeQuery(sql);
while (res.next()) {
System.out.println(res.getString(1) + "\t" + res.getString(2));
}
// load data into table
// NOTE: filepath has to be local to the hive server
// NOTE: /tmp/test_hive_server.txt is a ctrl-A separated file with two fields per line
String filepath = "/tmp/test_hive_server.txt";
sql = "load data local inpath '" + filepath + "' into table " + tableName;
System.out.println("Running: " + sql);
res = stmt.executeQuery(sql);
// select * query
sql = "select * from " + tableName;
System.out.println("Running: " + sql);
res = stmt.executeQuery(sql);
while (res.next()){
System.out.println(String.valueOf(res.getInt(1)) + "\t" + res.getString(2));
}
// regular hive query
sql = "select count(1) from " + tableName;
System.out.println("Running: " + sql);
res = stmt.executeQuery(sql);
while (res.next()){
System.out.println(res.getString(1));
}
}
}
I was able to create a table on the hive, but when I try to load data into the table, an error appears. so my question is, what should I put in "test_hive_server.txt" to make it work! Because I tried everything and every time I get the same error. Thank!
mistake:
Exception in thread "main" java.sql.SQLException: org.apache.thrift.TApplicationException: Internal error processing execute
at org.apache.hadoop.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:196)
at com.palmyra.nosql.HiveJdbcClient.main(HiveJdbcClient.java:52)
source to share
I found the answer myself:
ROW FORMAT DELIMITED FIELDS TERMINATED BY ' ';
define the format of the lines in the "file.txt" file that we will load into the table, and the default list of restrictions for files and fields:
\n
^A
^B
^C
press ^V^A could insert a ^A in Vim.
or you can just do:
create table tableName (key int, value string) ROW FORMAT DELIMITED FIELDS TERMINATED BY ' ';
and the file.txt should look like this:
value1 value2
value3 value4
in this example value1 and value3 represent the key and value4 and value5 represent the value
source to share