Copy a large list of files from one location to another
(using Python 3.4.2) I am trying to copy a list of over 8000 files in various locations to an intermediate directory. The original path will look something like D: \ dir \ img0 \ 000 and the destination path D: \ stage2 \ NW000.tif.
From what I understand Shutil can take care of this, I found a simple example on the internet that I was trying to adapt to work for my purpose. I saved 8000 entries in source and destination addresses in two separate text files with line breaks after each directory path. I want the src and dst variables to keep all the values of the lists and shutil.copy in order to copy the files to their new destination. I tried to test the code by creating 1 source folder and 3 dest. Folders: Test, TestA, TestB, TestC. I created two text files with a list of paths, this is how they look:
(source.txt)
C:\Users\user1\Desktop\Test\t1.txt
C:\Users\user1\Desktop\Test\t2.txt
C:\Users\user1\Desktop\Test\t3.txt
(dest.txt)
C:\Users\user1\Desktop\TestA\1t.txt
C:\Users\user1\Desktop\TestB\2t.txt
C:\Users\user1\Desktop\TestC\3t.txt
I saved two list text files, three dummy text files and a script in a test directory and executed the script, but I did not find any files in the target directories. I want to know why this script is not working (where to put the source, dest and script files for the script to execute correctly) .:
import shutil
import errno
file = open('source.txt', 'r')
source = file.readlines()
file = open('dest.txt', 'r')
destination = file.readlines()
def copy(src, dst):
try:
shutil.copytree(src, dst)
except OSError as e:
# If the error was caused because the source wasn't a directory
if e.errno == errno.ENOTDIR:
shutil.copy(src, dst)
else:
print('Directory not copied. Error: %s' % e)
copy(source, destination)
shutil.copytree(src, dst)
copies one directory tree from src
to dst
. It doesn't automatically iterate over file lists. You can try zipping the source and destination list and looping the resulting list:
for src, dst in zip(source, destination):
copy(src, dst)
You may also run into problems returning to shutil.copy(src, dst)
- this will fail if the target directory does not already exist.
source to share