Bash command to (re) move duplicate files [eg photos]
If I have a directory with a bunch of photos, and some of them are duplicates [in everything but the name], is there a way to get a list of uniques and move them to a different directory?
For example,
find . -type f -print0 | xargs -0 md5sum
which will give me a list of "md5 filename"
Now I just want to look at uniques based on this ... for example a pipe that matches sort -u
.
After that, I want to mv all those files somewhere else, but I might worry about that later ...
source to share
You can use fdupes :
fdupes -r .
to get a list of duplicates. This step should be possible with some command chain.
fdupes -r -f .
Shows only duplicated files. Therefore, if you have the image twice. You will get one entry instead of two duplicate paths.
To get around you can do:
for file in $(fdupes -r -f . | grep -v '^$')
do
mv "$file" duplicated-files/
done
But remember about name collisions.
source to share