Python-run script for multiple files
I have a python script that takes a filename as a command argument and processes that file. However, I have thousands of files that I need to process, and I would like to run a script for each file without having to add the file name as an argument each time.
The script works well when run in a separate file, for example:
myscript.py /my/folder/of/stuff/text1.txt
I have this code to do everything at once, but it doesn't work
for fname in glob.iglob(os.path.join('folder/location')):
proc = subprocess.Popen([sys.executable, 'script/location.py', fname])
proc.wait()
Whenever I run the above code, it doesn't throw an error, but it doesn't give me the intended output. I think the problem is that the script expects the path to the .txt file as an argument, and the code only gives it the folder the file is in (or at least the absolute link doesn't work).
How do I fix this problem?
source to share
If the files are in the same folder and if the script supports, you can use this syntax:
myscript.py /my/folder/of/stuff/*.txt
The wild card will be replaced with the corresponding files.
If the script doesn't support it, isolate the process like in this short example:
import sys
def printFileName(filename):
print filename
def main():
args = sys.argv[1:]
for filename in args:
printFileName(filename)
if __name__ == '__main__':
main()
Then, from the console, you can run it like this:
python MyScript.py /home/andy/tmp/1/*.txt /home/andy/tmp/2/*.html
This will print patches of all files in both folders.
Hope this can help.
source to share
You can write another script to do this. This is just a job, try usingos.walk
import sys, os
for root, dir, files in os.walk(PATH):
for file in files:
os.system ('myscript.py {}'.format(root + '\\' + file))
Provide the PATH
entire folder on os.walk
, it analyzes all the files in the directory.
If you want to parse specific files, say files with files for example .cpp
, then you can filter the filenames like this. add this afterfor file in files
if file.endswith('.cpp'):
source to share