Storing arrays in memory and then using those arrays
I am currently working on a program that requires preprocessing; filling multidimensional arrays of about 5765760 * 2.
My problem is that I have to run this preprocessing every time before I actually get to validate the data and it takes about 2 minutes.
I don't want to wait 2 minutes every time the test is run, but I also don't want to store the values in a file.
Is there a way to store the values in temporary memory rather than output them to a file?
source to share
I think what you are asking to translate is "can I make the JVM write data to some place in memory so that another JVM instance can later read on?"
And the simple answer is no , which is impossible.
When the JVM dies, the memory consumed by the JVM is returned to the OS. This stuff is gone .
So even the infamous sun.misc.Unsafe with DMA won't let you do that.
The only thing that would work is if your OS is Linux, you can create a RAM disk . And then you write your file.
So yes, you store your data in a file, but the file is in memory; thus read / write is much faster compared to disk IO. And this data remains available until you remove the RAM disk or reboot your OS.
On the other hand, when your OS is Linux and you have enough RAM (need to make a few GB!) Then you should just try if the "regular disk" is not good enough.
You see - these modern operating systems, they do a lot of things in the background. It might look like "writing to disk", but in the end, Linux is just using memory.
So, before you spend hours on fancy solutions, measure the impact of writing to disk!
source to share
Can I keep it in memory? Well, yes, this is already in memory! The obvious solution is to maintain your program. You can put your program in a repeatable loop - "enter Y to check again, or N to exit". Then your program can skip preprocessing if it's already done. Until you exit the program, you can do this as many times as you like.
Another thing you might be thinking is whether your code could be more efficient. If your code takes less time to run, it won't be annoying enough to wait for it. In general, if something can be done outside the loop, don't do it inside the loop. If you have an instruction executed five million times, it can add up. If it’s homework, you’ll probably use more time to make it more effective than you expected - but it’s not wasted time as you practice skills you may need later. Obviously I cannot give specific suggestions without code (and more efficient code is more efficient, probably better suited for Code Review update exchange site).
source to share