Controlling the Number of Background Jobs Running in the UNIX Shell
I have a problem when I run my bash command line scripts one by one, like:
./process.sh data1.txt ./process.sh data2.txt ... ./process.sh dataN.txt
What I would like to do is run them as background jobs, but you only have 10 runs at any given time. I can do this in a traditional programming language (very convenient with unions, wait style constructs), the question I have is how to do this using UNIX bash.
Thank.
+3
user1172468
source
to share
2 answers
BG_LIMIT=10
counter=0
for data in data*.txt; do
./process.sh $data &
if (( ++counter % $BG_LIMIT == 0 )); then
echo "Wating for background jobs to complete..."
wait
fi
done
echo "Waiting for any remaining jobs to complete..."
wait
+4
Costi ciudatu
source
to share
You can also use xargs
:
$ for x in data*.txt; do echo "$x"; done | xargs -n 1 -P 10 ./process.sh
Various cycle options are possible for
:, find
simple echo data*.txt
, etc. The task is to create a xargs
readable list of standard input filenames .
+2
chepner
source
to share