Clever Geek Handbook

Controlling the Number of Background Jobs Running in the UNIX Shell

I have a problem when I run my bash command line scripts one by one, like:

./process.sh data1.txt
./process.sh data2.txt
...
./process.sh dataN.txt

What I would like to do is run them as background jobs, but you only have 10 runs at any given time. I can do this in a traditional programming language (very convenient with unions, wait style constructs), the question I have is how to do this using UNIX bash.

Thank.

+3


source to share


2 answers 


BG_LIMIT=10
counter=0
for data in data*.txt; do
    ./process.sh $data &
    if (( ++counter % $BG_LIMIT == 0 )); then
        echo "Wating for background jobs to complete..."
        wait
    fi
done
echo "Waiting for any remaining jobs to complete..."
wait


+4


source


You can also use xargs

:

$ for x in data*.txt; do echo "$x"; done | xargs -n 1 -P 10 ./process.sh


Various cycle options are possible for

:, find

simple echo data*.txt

, etc. The task is to create a xargs

readable list of standard input filenames .

+2


source






More articles:


All Articles