Clever Geek Handbook

Use awk sed command and while loop to remove records from second file

I have two output files:

  • FILE-A contains over 70,000 unique records.
  • FILE-B contains a unique list that I need to remove from FILE-B.

FILE-A:

 TOM
 JACK
 AILEY
 BORG
 ROSE
 ELI

FILE-B content:

 TOM
 ELI

I want to remove everything listed in FILE-B from file-A.

FILE-C (results file):

 JACK
 AILEY
 BORG
 ROSE

I am guessing I need an operator while r for i

. Can someone help me with this? I need to cat

both read FILE-A and for every line in FILE-B I need to remove this from FILE-A.

Which command should you use?

+3


source to share


4 answers


You do not have any awk

, sed

or cycle. You just need grep

:

fgrep -vxf FILE-B FILE-A

Note the usage -x

for exact match of the entries.



Output:

JACK
AILEY
BORG
ROSE
+6


source


You can use grep -v -f

:



grep -xFvf FILE-B FILE-A
ACK
AILEY
BORG
ROSE
+4


source


If you are starting with sorted input, the tool for this task is comm 

comm -23 FILE-A FILE-B

option argument means

-2              suppress lines unique to FILE-B
-3              suppress lines that appear in both files

If not sorted at first you can do the following

comm -23 <(sort FILE-A) <(sort FILE-B)
+1


source


You don't need a loop, just one command awk

or sed

:

AWK:

awk 'FNR==NR {a[$0];next} !($0 in a)' FILE-B FILE-A >FILE-C

SED:

sed "s=^=/^=;s=$=$/d=" FILE-B | sed -f- FILE-A >FILE-C

Note:

  • As long as the sed version is running for the data shown, it will not process any text in FILE-B that could be interpreted as a regex pattern.
  • Awk's solution reads FILE-B completely into memory. He has no restriction on interpreting the text asregexas a solution sed

    .
+1


source






More articles:


All Articles