How to clean up after an incorrectly closed file object in the Python standard library (after an exception)

This is an interesting problem. As Lukas Graf pointed out in a comment, the problem is that the exception traceback object contains a reference to the call frame in which the exception was thrown. This call frame includes local variables that existed at the time, one of which is a link to an open file. So the object file still has a reference to it and is not closed properly.

For your stand-alone example, just remove try/except ConfigParser.Error

"works": Exception from file with invalid formats doesn't show up and stops the program. However, it catches an assertRaises

exception in your actual application to check that it is being tested. I'm not 100% sure why the trace persists even after a block with assertRaises

, but it seems to do it.

For your example, another promising solution is to change pass

in your proposal except

to sys.exc_clear()

:

try:
    config.read(path)
except ConfigParser.Error:
    sys.exc_clear()

      

        
        
        
      

    

This will save you the dull trace object and allow you to close the file.

However, it is not clear how to do this in your real application, because the offending sentence except

is inside unittest

. I think the easiest is not to use assertRaises

directly. Instead, write a helper function that runs your test, check for the exception you want, clean up the trick sys.exc_clear()

, and then add another custom exception. Then, complete the call to this helper method in assertRaises

. This way you gain control over the problematic exception thrown by ConfigParser and can properly clean it up (which unittest

it doesn't).

Here's a sketch of what I mean:

# in your test method
assertRaises(CleanedUpConfigError, helperMethod, conf_file, malformed_conf_file)

# helper method that you add to your test case class
def helperMethod(self, conf_file, malformed_conf_file):
     gotRightError = False
     try:
          or6 = OptionReader(
               config_files=[conf_file, malformed_conf_file],
               section='sec',
          )
     except ConfigParser.Error:
          gotRightError = True
          sys.exc_clear()
     if gotRightError:
          raise CleanedUpConfigError("Config error was raised and cleaned up!")

      

        
        
        
      

    

I haven't tested this of course, because I don't have the whole setup tuned to your code. You may need to tweak it a bit. (Think about it, you might not even need exc_clear()

it if you do this, because since the exception handler is now in a separate function, the traceback should be properly cleaned up when it helperMethod

exits.) However, I think this idea might lead you somewhere. Basically you need to make sure that the sentence except

that catches this particular ConfigParser.Error

one is written by you so that it can be cleaned up before attempting to delete the test file.

Addendum: It seems that if the context manager handles the exception, the traceback actually persists until the function containing the block with

ends, as shown in this example:

class CM(object):
    def __init__(self):
        pass

    def __enter__(self):
        return self

    def __exit__(self, exc_type, exc_value, tb):
        return True

def foo():
    with CM():
        raise ValueError
    print(sys.exc_info())

      

        
        
        
      

    

Even though the block has with

completed by the time print

it was thrown, so exception handling must be completed, sys.exc_info

it still returns information about the exception as if there was an active exception. This is what happens in your code too: the block with assertRaises

causes the trace to persist until the end of that function, interfering with yours os.remove

. This seems like buggy behavior and I notice that it no longer works that way in Python 3 ( print

prints (None, None None)

), so I think it was a wart fixed with Python 3.

Based on this, I suspect it might be enough to just insert sys.exc_clear()

right before yours os.remove

(after the block with assertRaises

).