Help Needed Bash Script Execution
I have a script that extracts comma information from a file and prepares an update statement. The file will look like this:
ID,NAME,DATE,TIME,HOURS,EMPNUMBER 1, Joe, 12/11, 12:45, 5, 333 2, John, 12/12, 16:45, 7, 666
My script takes a file as a parameter and runs from the command line as shown below:
./runScript.sh file.csv
My script code is below:
for i in ` cat $1 | grep -v "EMPNUMBER" | cut -d',' -f4,5`
do
time=`echo $i | cut -d',' -f1`
hours=`echo $i | cut -d',' -f2`
echo "update jobs.j j set j.time= $time where j.hours=$hours;"
done
I'm just wondering why when I run my script it skips the top line in my file, which is the header information. Obviously this is the desired effect, but in order for my learning to progress I need to understand why the first line is being skipped from the file.
Can anyone help with my understanding? ~
source to share
If you are learning bash, in addition to explaining what grep -v "EMPNUM"
is what causes the header to be skipped (the parameter -v
means the search strings are not included EMPNUM
), there are several other points to indicate. First, good bash code uses bash tools, which allows you to read input and parsing data, rather than relying on spawning subshells to run additional programs (i.e. cat, grep, cut
).
Note: There is nothing wrong with using it cat, grep, cut
, but while recognizing that bash itself provides tools that do exactly what you use, these 3 other programs will help strengthen your programming skills.
First, bash provides an inline read
for reading data from stdin
or any other file. To read a file, you usually see while read var1 var2; do... done <"filename"
instead for i in $(cat file)
- for many reasons. Further, instead of calling cut...
, bash provides parameter expansion/substring extraction
to handle parsing any line of text into any individual variables. Also, by choosing variables to accompany read
intelligently, you can opt out of use entirely substring extraction
.
The following illustrates the use of bash alternatives for the approach cat, grep, cut
shown in your example. If you are interested in learning bash
, give it a look and let me know if you have any questions. You can use echo
and printf
interchangeable for output. While echo
generally simpler, it printf
provides several benefits. It's worth exploring both ...
#!/bin/bash
## set the datafile name (defaults to 'dat/empdata.dat')
dfn="${1:-dat/empdata.dat}"
## validate that file is readable
[ -r "$dfn" ] || {
printf "\n error: file not readable '%s'. Usage: %s [filename (dat/empdata.dat)]\n\n" "$dfn" "${0//*\//}"
exit 1
}
## simple output header for data
printf "\nEmployee data read from file: '%s'\n\n" "$dfn"
## read each line in file, skipping header (where $id = ID)
# IFS is set to include ',' in addition to default ' \t\n'
while IFS=$' ,\t\n' read -r id nm dt tm hrs eno || [ -n "$hrs" ]; do
# if header row - skip
[ "$id" = "ID" ] && continue
# print out each of the values for the employee
printf "ID: %s NAME: %-4s DATE: %s TIME: %s HOURS: %s EMPNUMBER: %s\n" \
"$id" "$nm" "$dt" "$tm" "$hrs" "$eno"
done <"$dfn"
input file:
$ cat dat/empdata.dat
ID,NAME,DATE,TIME,HOURS,EMPNUMBER
1, Joe, 12/11, 12:45, 5, 333
2, John, 12/12, 16:45, 7, 666
output:
$ bash empdata.sh
Employee data read from file: 'dat/empdata.dat'
ID: 1 NAME: Joe DATE: 12/11 TIME: 12:45 HOURS: 5 EMPNUMBER: 333
ID: 2 NAME: John DATE: 12/12 TIME: 16:45 HOURS: 7 EMPNUMBER: 666
source to share
using awk i tried
awk -F ',' '{if(NR==1) for(i=1;i<=NF;i++) a[i]=$i}{if(NR>=2)for(i=1;i<=NF;i++) printf("%s:%s\t",a[i],$i)}{printf("\n")}' file.txt
output:
ID:1 NAME: Joe DATE: 12/11 TIME: 12:45 HOURS: 5 EMPNUMBER: 333
ID:2 NAME: John DATE: 12/12 TIME: 16:45 HOURS: 7 EMPNUMBER: 666
source to share