WindowsError: [Error 3] The system cannot find the path specified (when is the path too long?)
WindowsError: [Error 3] The system cannot find the path specified (when is the path too long?)
I am creating a script to find unique files between two directories. For this I use os.walk()
to view files, and if there are files of the same size, I keep them in mind so that they are the same (opening files in the process). The problem is that some files cause the above error when opened. The most common reason people run into this problem is because the path is not correctly connected, thus creating a script to try to open a file that does not exist. This does not apply to me.
After using different combinations of directories, I started to notice a pattern where the files that generate the error seem to have deep directory structure and long filename. I can't think of any other reasons for the problem - character encoding errors (I don't decode all my paths to UTF-8) and the paths exist by virtue os.walk()
.
My code:
for root, dirs, files in os.walk(directory):
for filename in files:
file_path = os.path.join(root, filename)
My hashing code:
def hash(file_path):
with open(dir_file, 'rb') as f:
hasher = hashlib.md5()
while True:
buf = f.read(byte_size)
if buf != '':
hasher.update(buf)
else:
break
result = hasher.hexdigest()
return result
Edit: The most recent path where the problem occurred is 5 deep directories (containing 142 characters, which requires a double backslash) and the filename is another 122 characters
source to share
This is due to the size limitation of the Windows API file path as described on MSDN :
In the Windows API (with some exceptions, described in the following paragraphs), the maximum length for a path is MAX_PATH , which is defined to be 260 characters . The local path is structured in the following order: drive letter, colon, backslash, backslash-separated name components, and a terminating null character. For example, the maximum path on drive D is "D: \ some 256-character path string", where "" represents an invisible terminating null character for the current system code page. (The <> characters are used here for visual clarity and cannot be part of a valid path string.)
As explained on this page, newer versions of Windows support the extended file path prefix ( \\?\
) used for Unicode paths, etc., but this is not consistent or guaranteed behavior, i.e. this does not mean that it will work in all cases.
Anyway, try adding your extended path prefix and see if it works for your case:
file_path = "\\\\?\\" + os.path.join(root, filename)
source to share