Monday, April 29, 2024
79
rated 0 times [  79] [ 0]  / answers: 1 / hits: 42018  / 3 Years ago, mon, may 31, 2021, 11:17:24

So I'm checking the md5 hash of my files with this as my output:



657cf4512a77bf47c39a0482be8e41e0  ./dupes2.txt
657cf4512a77bf47c39a0482be8e41e0 ./dupes.txt
8d60a927ce0f411ec94ac26a4785f749 ./derpina.txt
15f63928b8a1d5337137c38b5d66eed3 ./foo.txt
8d60a927ce0f411ec94ac26a4785f749 ./derp.txt


However, after running find . -type f -exec md5sum '{}' ';' | uniq -w 33 to find the unique hashes I get this:



657cf4512a77bf47c39a0482be8e41e0  ./dupes2.txt
8d60a927ce0f411ec94ac26a4785f749 ./derpina.txt
15f63928b8a1d5337137c38b5d66eed3 ./foo.txt
8d60a927ce0f411ec94ac26a4785f749 ./derp.txt


From my understanding, only one of either derpina.txt or derp.txt should be showing up since their hashes are the same. Am I missing something? Can anyone enlighten me as to why it outputs like this?


More From » command-line

 Answers
2

You need to use sort before uniq:


find . -type f -exec md5sum {} ';' | sort | uniq -w 33

uniq only removes repeated lines. It does not re-order the lines looking for repeats. sort does that part.


This is documented in man uniq:



Note: uniq does not detect repeated lines unless they are adjacent. You may want to sort the input first, or use sort -u without uniq.



[#22841] Wednesday, June 2, 2021, 3 Years  [reply] [flag answer]
Only authorized users can answer the question. Please sign in first, or register a free account.
ingsta

Total Points: 391
Total Questions: 103
Total Answers: 124

Location: Bonaire
Member since Wed, Mar 29, 2023
1 Year ago
ingsta questions
Sun, Oct 23, 22, 01:42, 2 Years ago
Sat, Oct 30, 21, 11:27, 3 Years ago
Sun, Nov 28, 21, 12:49, 2 Years ago
;