Distributed data visited on a Storm cluster and how does MemoryMapState store distributed data?
I am trying to design a topology that can have some so called "globals" just like the Distributed caching in storm question . I know it's better to use some sort of distributed cache store like Memcache or Redis, but I still have a few questions:
-
What's the best way to use data caching? It looks like I shouldn't use caching directly in the function
execute
ornextTuple
should I spawn a thread in the spout / bolt to periodically receive this data? -
MemoryMapState
is a "state" used to store a permanent result in a tedent storm. He said heMemoryMapState
usesConcurrentHashMap
caching for storing data as the codes inclass MemoryMapState
show belowstatic ConcurrentHashMap<String, Map<List<Object>, Object>> _dbs = new ConcurrentHashMap<String, Map<List<Object>, Object>>();
So how can you
ConcurrentHashMap
store and query data among different machines in a Storm cluster?
source to share
No one has answered this question yet
See similar questions:
or similar: