How can I replace a line in the same file with SED in Unix Shell scripts?
In the link to this question After getting the line id match in the first and second file, I need to replace the line in the first file with the line in the second file. For what I am using SED as shown below. But Sed only replaces that line in the new file. How can I achieve an update in the same file without a temp file (because these are very large files).
#!/bin/ksh
while read line
do
var=`echo $line|cut -c 3-25`
while read i
do
var1=`echo $i|cut -c 3-25`
if [ $var == $var1 ];
then
sed -i s/$line/$i/ t1
else
echo "not matched"
fi
done < t2
done < t1
Even without the option -i
also I don't get the result. Please help me.
Edit . Or can you suggest me any other optimal way to do it without temporary files or with any scripting languages (less preferred).
source to share
Cannot be used sed
. Since your script is now standing, this is what it says:
- For each line in t1
- Step through all the lines in t2
- If one of the lines at t2 matches the current line at t1, step all lines at t1 and replace the matches
- Go to next line in t1 and repeat
This means that the entire file t2 is read every time one line is read from t1. This is incredibly ineffective.
No need to use echo
and cut
substring. In Bash and ksh, you can:
var=${line:3:23}
Note: cut uses character positions for the start and end of the range, while this shell construct uses a start position and number of characters, so you need to adjust the numbers accordingly.
If t2 is a list of replacements to be done in t1, so t2 is a "script" of sorts, then this might do what you want:
keystart=3
keylen=23
while read line
do
var="${line:$keystart:$keylen}"
if (( ${#var} == keylen )) # need "$" then don't need "$"
then
sed -in "/^.\{$keystart\}$var/ c$line" t1 # use double-quote so vars get expanded
fi
done < t2
This will find all the rows in t1 so that every row in t2 matches and does the replacement.
If, however, t1 and t2 have a string match for a string, and you only want to do the replacement where the matching strings match, then this, using a temporary file, would be most efficient:
tempfile=$(mktemp)
keystart=3
keylen=23
while read line1
do
var1="${line1:$keystart:$keylen}"
read line2 <&3 # use file descriptor 3 for input
var2="${line2:$keystart:$keylen}"
if [[ $var1 == $var2 && ${#var2} == $keylen ]]
then
echo "${line2}" >> $tempfile # substitute when matched
else
echo "${line1}" >> $tempfile # keep when not matched
fi
done < t1 3< t2 # t1 is input on stdin, t2 is input on fd 3
mv t1 t1.bak && mv $tempfile t1
source to share
You cannot replace one string with another. As the lines are usually of different lengths and will overlap. If all lines in your file are the same length, they might work. I also suggest that you use a more appropriate language for this task (eg Perl), because it will be really complex code in Shell. I think that you should look for a solution with temporary files because it will be easier to implement and can be easily debugged. Imagine what you would do if your huge file is uploaded due to an error in the script.
source to share
Sed is designed to work in a pipeline - hence the name "Stream EDitor". You can use ex script to edit the file instead. Ex is the line-based text editor that vi was originally based on (not as old as ed, old bear fonts and stone knives text editor, but almost). A simple example that you can modify for your purpose could be as follows:
ex t1 << EOF
$lineNum
s/^.*$/$newline/
w
q
EOF
This script first goes to the line denoted by $ lineNum, replaces the beginning of the entire line (^) with the end ($) with the contents of the line $ newline, then writes and exits. These commands are surrounded by " <<EOF
" and " EOF
" which make up the "here" document, essentially setting up the script commands as stdin.
source to share