Apache beam bigtable Iterable mutation
I am migrating my javascript java 1.9 in google 2.0 to pool 2.0 and I am trying to use BigtableIO.Write
....
.apply("", BigtableIO.write()
.withBigtableOptions(bigtableOptions)
.withTableId("twoSecondVitals"));
In ParDo, before BigtableIO, I am struggling to make an Iterable.
try{
Mutation mutation = Mutation.parseFrom(new ObjectMapper().writeValueAsBytes(v));
Mutation mu[] = {mutation};
Iterable<Mutation> imu = Arrays.asList(mu);
log.severe("imu");
c.output(KV.of(ByteString.copyFromUtf8(rowKey+"_"+v.getEpoch()), imu));
}catch (Exception e){
log.severe(rowKey+"_"+v.getEpoch()+" error:"+e.getMessage());
}
The above code throws the following InvalidProtocolBufferException: The target protocol message group tag does not match the expected tag
v - list of objects (Vitals.class). Hbase api uses Put method to create mutation. How do I create a BigTable mutation that will work with the BigtableIO sink?
+3
source to share
1 answer
Looking through the sdk tests I was able to find my answer.
Iterable<Mutation> mutations =
ImmutableList.of(Mutation.newBuilder()
.setSetCell(
Mutation.SetCell.newBuilder()
.setValue(ByteString.copyFrom(new ObjectMapper().writeValueAsBytes(v)))
.setFamilyName("vitals")
).build());
+4
source to share