Reservation of orders and calculation of orders
I am working on creating an Order Level 2 book at work.
I am using a parallel dictionary to store data for bids and query separately.
I am reading packet data using a UDP listener, and after processing the packet, I push the update to the bid / ask dictionary for the task.
This works great and updates are fast, but it seems clunky.
When I need to do any calculations on the Order Log, I have to block it to do the calculation, which seems bad given its parallel dictionary. I feel like I'm slowing down everything that defeats the target.
How would you advise me to work around blocking issues and slow down updates to calculate data?
Sorry, I can't post the code as I am currently not working, but I just can't stop thinking about how to solve this.
source to share
The design reached its limits with only one data silo. Therefore, I recommend that the design be redesigned to accommodate two silos and a data manager.
The main goal should be to do the settlement operations required in a timely but secure manner, and then secondly focus on any data entry that should be free so as not to be blocked.
This project has three streams, one manager and two data silos.
Data input
The stream for enqueueUDP listener
will store all incoming data in the ConcurrentQueue Class . This store will process data concurrency data jobs by FIFO method.
Data manager
The second thread runs a singleton of the data manager, which will smurf ( TryDequeue ) the data from the data entry ConcurrentQueue
to the final silo.Since the incoming data is in the queue, the total number of records can be throttled to cross this virtual bridge in order to achieve standard performance that can be measured / to control.
Final data silo ( Orderbook
)
The final flow for summing operations is where the data will ultimately reside. The data itself can be a standard vocabulary that will be locked by either the data manager or the summarization process. You can then change the priority of access in favor of the summation process via the data manager or vice versa; which will be determined by the final needs of the operation.
With this design, data entry is unlocked and the addition operation takes precedence over data entry in a measured manner.
source to share