The memory allocated by the IO function is not released even after calculating the value

Let's say we need to perform a series of complex I / O operations, each of which takes up a lot of space, but returns a fairly simple value such as Int or String. In the real world that might be https for example receive requests. I will be referring to an action of the type hardIO

here.

page <- hardIO

      

Naturally, as the program progresses along this line, we would like all memory to hardIO

be freed except for a small bit in which we store the actual value. There is no reason to keep large amounts of data only needed to perform an action and calculate the result.

I noticed that this is not the case. In fact, it page

continues to take up a lot of space until after the last link to it. For example, if I want to print the result of execution hardIO

at the end of the program, until then the memory will not be freed. No memory is freed at all! In fact, it may even take up twice as much memory after executing it as the maximum it took up during execution.

page <- hardIO
-- A lot of memory not getting freed!
print page

      

When we start to linearly render many pages, space will be wasted linearly excessively. I believe that the algorithm should work in constant time, except for a linearly growing, but still relatively small list of results.


I am trying to write a program that repeatedly receives certain https API endpoints and about the hundredth request, it ends up eating RAM, starts changing and I have to kill the process. (Also: I think the copy garbage collector doesn't go well with replacement.) This is very frustrating. You can view the actual code here (commit fe0ebcb should work more or less).

In an attempt to solve the problem, I put together a simplified program that demonstrates the essence of the problem. If we find a solution for the simplified case, the general solution will certainly follow immediately. You can view the simplified program here . (I've included the script I'm using to generate it and get the profiling results.) The memory profiling graph for the cost center hardIO

shows us that in the moment between binding and releasing its return value, a lot of space is occupied by untagged gray stuff. I guess we need to get rid of.

+3


source to share





All Articles