Memory not returning after calling python function

I have a function that parses a sentence, creating a chart. But Python holds on to what memory was allocated when that function was called. That is, I do

best = translate(sentence, grammar)


and somehow my memory rises and remains. Here's the function:

from string import join
from heapq import nsmallest, heappush
from collections import defaultdict

MAX_TRANSLATIONS=4 # or choose something else

def translate(f, g):
    words = f.split()
    chart = {}
    for col in range(len(words)):
        for row in reversed(range(0,col+1)):
            # get rules for this subspan                                        
            rules = g[join(words[row:col+1], ' ')]
            # ensure there at least one rule on the diagonal                  
            if not rules and row==col:
                rules=[(0.0, join(words[row:col+1]))]
            # pick up rules below & to the left                                 
            for k in range(row,col):
                if (row,k) and (k+1,col) in chart:
                    for (w1, e1) in chart[row, k]:
                        for (w2, e2) in chart[k+1,col]:
                            heappush(rules, (w1+w2, e1+' '+e2))
            # add all rules to chart                                            
            chart[row,col] = nsmallest(MAX_TRANSLATIONS, rules)
    (w, best) = chart[0, len(words)-1][0]
    return best

g = defaultdict(list)
g['cela'] = [(8.28, 'this'), (11.21, 'it'), (11.57, 'that'), (15.26, 'this ,')]
g['est'] = [(2.69, 'is'), (10.21, 'is ,'), (11.15, 'has'), (11.28, ', is')]
g['difficile'] = [(2.01, 'difficult'), (10.08, 'hard'), (10.19, 'difficult ,'), (10.57, 'a difficult')]

sentence = "cela est difficile"
best = translate(sentence, g)


I am using Python 2.7 on OS X.


source to share

2 answers

Inside a function, you set an rules

element grammar

; rules

then refers to that element, which is a list. Then you add items in rules

with heappush

, which (since lists are mutable) means that it is grammar

holding the pressed values ​​across that list. If you don't want this to happen, use copy

in assignment rules

or deepcopy

in grammar at the beginning translate

. Note that even if you copy the list to rules

, the grammar will write an empty list every time you retrieve an element for the missing key.



Try to run gc.collect

after running this function.



All Articles