Phoenix Upsert Delays
I am using the following line to update data in Hbase using Phoenix.
upsert into fraud_xxx_y.detection_details (call_date_hour, msisdn,
insert_time, call_time, source, shutdown_file,imsi, imei, cell_id, lac,
vendor, model, calls, cdt_duration ) values ( '2017032121' ,
'1117037044082', to_date('2017-03-21 21:38:32.752'), '2017-03-21 03:13:58','GKP','SbGSM_20170321172419_269.txt','621300303944920','null','50007'
,'0','null','null',108,269)
The same upsert statement is used for two different databases, db1 and db2, and both have the same detect_details tables.
We see that for db1, only for the detect_details table, the upsert is delayed by an hour each time, while for db2, the upsert is delayed immediately.
The only differences in the two processes are for db1, upserts are close to 100K for 10 minutes, and for db2 it is 10K.
It seems that for db1 it keeps waiting for about an hour with data in some cache and at the end of the hour wakes up and puts all the data into a table. We haven't configured a cache for the operation, so latency is a real mystery. Appreciate any insight into this.
source to share
No one has answered this question yet
Check out similar questions: