Clever Geek Handbook

How to find duplicate lines in a connection text file

I recently converted 10 JavaScript files into one file, which I run the JavaScript compiler into. I just had an error when I reused the function name.

Is there a tool to check for duplicate strings / function names in a merged file?

Or do I need to create a small program?

0


source to share


2 answers


I haven't tried this, but "Dupli Find" available at http://www.rlvision.com/dupli/about.asp may help you.

The powershell script window outlined at http://secretgeek.net/ps_duplicates.asp also helps you write your own tool.



There is also a scripting solution at http://www.microsoft.com/technet/scriptcenter/resources/qanda/aug05/hey0819.mspx

+2


source 


cat file.js | grep -o "function\([[:space:]]\+[a-zA-Z0-9_]\+\)\?[[:space:]]*(" | sort | uniq -c | sort -n

What reads:

  • Cat file
  • Find function definitions (function-ws-name-ws-paren) -o only fetches the relevant parts of the lines (i.e. only the definitions themselves)
  • sort (for next step)
  • Read the same sequential lines (uniq removes duplicates, -c adds count)
  • sort by number of matches so that duplicates (if they appear last)


You can filter out non-duplicates, but it's easier to just sort so they come first.

Edit Modified regex to include anonymous functions

0


source






More articles:


All Articles