BigInteger memory leak causes stack overflow in Java

I am trying to write an optimized fibonacci as an assignment to be able to compute fib (300) and fib (8000). Here is what I have so far (map is a HashMap).

public static BigInteger fibMemo(int n) {

    if (n <= 1){
        return BigInteger.valueOf(n);
    }

    if (!map.containsKey(n)){
        map.put(n, fibMemo(n-1).add(fibMemo(n-2)));
    }
    return map.get(n);        
}

      

When i call

System.out.println("300: " + FibonacciMemo.fibMemo(300));

      

by itself, it works fine. Besides,

System.out.println("8000: " + FibonacciMemo.fibMemo(8000));

      

works great if I comment out the previous call to fib (300). But if I keep both calls, I get a stack overflow on the recursive fibMemo. This is very strange to me. Can anyone please clarify the situation? Thanks in advance.

Here is the code:

import java.util.HashMap; // Import Java HashMap so we can use it
import java.math.BigInteger;

public class FibonacciMemo {
    private static HashMap<Integer, BigInteger> map = new HashMap<Integer, BigInteger>();
    /**
     * The classic recursive implementation with no memoization. Don't care
     * about graceful error catching, we're only interested in performance.
     * 
     * @param n
     * @return The nth fibonacci number
     */
    public static int fibNoMemo(int n) {
        if (n <= 1) {
            return n;
        }
        return fibNoMemo(n - 2) + fibNoMemo(n - 1);
    }
    /**
     * Your optimized recursive implementation with memoization. 
     * You may assume that n is non-negative.
     * 
     * @param n
     * @return The nth fibonacci number
     */
    public static BigInteger fibMemo(int n) {
        // YOUR CODE HERE
        if (n <= 1){
            return BigInteger.valueOf(n);
        }

        if (!map.containsKey(n)){
            map.put(n, fibMemo(n-1).add(fibMemo(n-2)));
        }
        return map.get(n);        
    }
public static void main(String[] args) {
        // Optional testing here        
        String m = "Fibonacci real name was Leonardo Pisano Bigollo.";
        m += "\n" + "He was the son of a wealthy merchant.\n";
        System.out.println(m);
         System.out.println("300: " + FibonacciMemo.fibMemo(300));
        System.out.println("8000: " + FibonacciMemo.fibMemo(8000));
        // 46th Fibonacci = 1,836,311,903
        // 47th Fibonacci = 2,971,215,073
    }
}

      

+3


source to share


4 answers


There are two problems with the code. The consumption of the stack is obvious. Memobilization reduces the time complexity from exponential to linear, but still the method has a linear stack consumption - for an input value of 8000, it allocates 8000 stack frames.

As stated in the docs , the default stack size for a stream is 320K , which is enough for roughly 1000 - 2000 frames, which is not enough. You can increase the stack size with a -Xss

JVM switch , but that's not bullet proof yet. You should use an iterative implementation instead.



The second problem is that your static cache is never cleared, which basically causes a memory leak. You can either wrap the recursive method in another that will clear the hashmap after the recursion completes, but this will throw a little performance back because the results of one call cannot be reused in the next.

A more efficient solution would be to use a proper cache implementation, which does not require manual flushing, but itself limits size and garbage collection. Guava provides such an implementation.

+2


source


It looks like your recursive algorithm is too high for the default Java stack size. The stack memory is optimized differently than the heap in hardware, and you shouldn't use algorithms with this kind of recursion. Some languages ​​can optimize tail recursion. It looks like Java won't always optimize your code in this case.

So imo's best solution is to just rewrite the code to use loops instead.



   private final static List<BigInteger> fibs = new ArrayList<>();
   static{ fibs.add( BigInteger.ZERO ); fibs.add( BigInteger.ONE ); }

   public static BigInteger lFib( int n ) {
      if( n < 0 ) throw new IllegalArgumentException();
      if( n >= fibs.size() ) {
         for( int i = fibs.size(); i <= n; i++ )
            fibs.add( fibs.get(i-2).add( fibs.get(i-1) ) );
      }
      return fibs.get(n);
   }

      

Very poorly tested.

+1


source


The problem is the size of the thread stack, which can exhaust a large number of recursive calls. The solution is to ensure that the stack is large enough. You can try to run your application using vm arg -Xss

. I tried with -Xss2m

and it worked fine.

0


source


Change your code

        map.put(n, fibMemo(n-1).add(fibMemo(n-2)));

      

to

        map.put(n, fibMemo(n-2).add(fibMemo(n-1)));

      

It works great.

First calling sequences

fibMemo(10) nested level = 0
fibMemo(9) nested level = 1
fibMemo(8) nested level = 2
fibMemo(7) nested level = 3
fibMemo(6) nested level = 4
fibMemo(5) nested level = 5
fibMemo(4) nested level = 6
fibMemo(3) nested level = 7
fibMemo(2) nested level = 8
fibMemo(1) nested level = 9
fibMemo(0) nested level = 9
fibMemo(1) nested level = 8
fibMemo(2) nested level = 7
fibMemo(3) nested level = 6
fibMemo(4) nested level = 5
fibMemo(5) nested level = 4
fibMemo(6) nested level = 3
fibMemo(7) nested level = 2
fibMemo(8) nested level = 1

      

Latest calling sequences

fibMemo(10) nested level = 0
fibMemo(8) nested level = 1
fibMemo(6) nested level = 2
fibMemo(4) nested level = 3
fibMemo(2) nested level = 4
fibMemo(0) nested level = 5
fibMemo(1) nested level = 5
fibMemo(3) nested level = 4
fibMemo(1) nested level = 5
fibMemo(2) nested level = 5
fibMemo(5) nested level = 3
fibMemo(3) nested level = 4
fibMemo(4) nested level = 4
fibMemo(7) nested level = 2
fibMemo(5) nested level = 3
fibMemo(6) nested level = 3
fibMemo(9) nested level = 1
fibMemo(7) nested level = 2
fibMemo(8) nested level = 2

      

The latter is less than the consumption of the stack.

0


source







All Articles