Hbase batch get and SocketTimeoutException
I am using java and I want to do batch like this.
final List<Get> gets = uids.stream()
.map(uid -> new Get(toBytes(uid)))
.collect(Collectors.toList());
Configuration configuration = HBaseConfiguration.create();
conf.set("hbase.zookeeper.quorum", quorum);
conf.set("hbase.zookeeper.property.clientPort", properties.getString("HBASE_CONFIGURATION_ZOOKEEPER_CLIENTPORT"));
conf.set("zookeeper.znode.parent", properties.getString("HBASE_CONFIGURATION_ZOOKEEPER_ZNODE_PARENT"));
HTable table = new HTable(configuration, tableName);
return table.get(gets);
When the list has 10K, everything is fine.
When I try to make 100K gets in one batch, I have an exception:
java.lang.RuntimeException: org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 100000 actions: SocketTimeoutException: 100000 times,
Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 100000 actions: SocketTimeoutException: 100000 times,
at org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:203) ~[hbase-query-layer-r575958b.jar:?]
at org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$500(AsyncProcess.java:187) ~[hbase-query-layer-r575958b.jar:?]
at org.apache.hadoop.hbase.client.AsyncProcess.getErrors(AsyncProcess.java:922) ~[hbase-query-layer-r575958b.jar:?]
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:2402) ~[hbase-query-layer-r575958b.jar:?]
at org.apache.hadoop.hbase.client.HTable.batchCallback(HTable.java:868) ~[hbase-query-layer-r575958b.jar:?]
at org.apache.hadoop.hbase.client.HTable.batchCallback(HTable.java:883) ~[hbase-query-layer-r575958b.jar:?]
at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:858) ~[hbase-query-layer-r575958b.jar:?]
at org.apache.hadoop.hbase.client.HTable.get(HTable.java:825) ~[hbase-query-layer-r575958b.jar:?]
at hbase_query_layer.hbase.HbaseConnector.get(HbaseConnector.java:89) ~[hbase-query-layer-r575958b.jar:?]
... 15 more
What's wrong?
Also, I saw (in the web interface) that the request to the region server (where the table is stored) is growing (batch size is 100k, after a few minutes I saw that the number of requests is 700k and still growing, but only my clien write sth this table).
Also, on the hbase register server I saw in hbase-hbase-regionserver.out
Exception in thread "RpcServer.handler=25,port=60020" java.lang.StackOverflowError
at org.apache.hadoop.hbase.CellUtil$1.advance(CellUtil.java:203)
at org.apache.hadoop.hbase.CellUtil$1.advance(CellUtil.java:203)
How to fix it?
+3
source to share
1 answer
I found the issue: https://issues.apache.org/jira/browse/HBASE-11813
Unfortunately I have HBase version 0.98.0.2.1.1.0-385-hadoop2, so I need to create chunks like:
final List<List<Increment>> batchesToExecute = chopped(increments, conf.getBatchIncrementSize());
static <T> List<List<T>> chopped(List<T> list, final int L) {
List<List<T>> parts = new ArrayList<>();
final int N = list.size();
for (int i = 0; i < N; i += L) {
parts.add(new ArrayList<>(list.subList(i, Math.min(N, i + L))));
}
return parts;
}
+1
source to share