Distributed data visited on a Storm cluster and how does MemoryMapState store distributed data?

I am trying to design a topology that can have some so called "globals" just like the Distributed caching in storm question . I know it's better to use some sort of distributed cache store like Memcache or Redis, but I still have a few questions:

  • What's the best way to use data caching? It looks like I shouldn't use caching directly in the function execute

    or nextTuple

    should I spawn a thread in the spout / bolt to periodically receive this data?

  • MemoryMapState

    is a "state" used to store a permanent result in a tedent storm. He said he MemoryMapState

    uses ConcurrentHashMap

    caching for storing data as the codes in class MemoryMapState

    show below

    static ConcurrentHashMap<String, Map<List<Object>, Object>> _dbs = 
                new ConcurrentHashMap<String, Map<List<Object>, Object>>();
    
          

    So how can you ConcurrentHashMap

    store and query data among different machines in a Storm cluster?

+3


source to share





All Articles