How to close files in python when I have no file ids
I was unable to open new files in Python. When I looked with help ls -l /proc/PID/fd
, I saw a lot of files open to the Python process. The module I am using appears to open a lot of files and not close them.
I expected to be able to close the files by deleting the objects associated with the module that opened the files, but nothing happened.
I also expected to see file objects somewhere in the garbage collection, but I haven't seen anything that looks like open files with this:
for obj in gc.get_objects():
if hasattr(obj, 'read'):
print(obj)
Files disappear when exiting Python.
source to share
Probably the problem is that these file descriptors are leaking without being tied to Python objects. Python has no way of seeing actual file descriptors (OS resources) that are not associated with Python objects. If they were associated with Python objects, Python would close them when they were garbage collected. In addition, the third party library does its own file descriptor tracking.
You can use os.close
for prime integers to close the associated file descriptor. If you know what file descriptors you want to open (usually stdin / stdout / stderr, which are 0, 1, and 2, and maybe a few others), you can just close all the other integers between 0 and 65535, or just those in /proc/<pid>/fd
:
import os
KEEP_FD = set([0, 1, 2])
for fd in os.listdir(os.path.join("/proc", str(os.getpid()), "fd")):
if int(fd) not in KEEP_FD:
try:
os.close(int(fd))
except OSError:
pass
This is a pretty nasty hack. Better solution would be to fix a third party library.
source to share