Find and replace text in a large 47 GB file
I need to find and replace tasks on a fairly large file of about 47 GB in size.
Does anyone know how to do this? I've tried using services like TextCrawler, EditpadLite and more, but nothing supports this large file.
I assume it can be done via the command line.
Do you have any idea how this can be done?
Sed (stream editor for filtering and transforming text) is your friend.
sed -i 's/old text/new text/g' file
Sed performs transformations of text in one pass.
I am using FART - Find and replace the text for Lionello Lunesu.
It works fine on Windows Seven x64.
You can find and replace text using the following command:
fart -c big_filename.txt "find_this_text" "replace_to_this"
On Unix or Mac:
sed 's / oldstring / newstring / g' oldfile.txt> newfile.txt
fast and easy...
If you are using a Unix-like system, you can use cat | sed do it
cat hosted_domains.txt | sed s/com/net/g
The example replaces com with the net in the list of domain names, and then you can pipe the output to a file.
For me, none of the tools suggested here work. Textcrawler ate all the computer memory, SED didn't work at all, Editpad complained about memory ...
Solution: Create your own script in python, perl, or even C ++.
Or use the PowerGrep tool, which is the easiest and fastest option.
I haven't tried farting, it's only command line and maybe not very friendly.
Some hex editors, like Ultraedit, also work well.
With sed (a stream editor for filtering and text trasformation), it does the transformation of the text in one pass.
sed -i 's/old/new/g' bigfile.txt
I used
sed 's/[nN]//g' oldfile.fasta > newfile.fasta
to replace all n instances in my 7Gb file.
If I omitted the aspect > newfile.fasta
it took an age as it scrolled the screen showing me every line of the file.
With > newfile
it launched it in seconds on ubuntu server