In this post I am sharing a small tip which is very useful while for system admins. The tip is about how to print duplicate lines from file. Sometimes we get the big file with lot of contents.And there may be chances of duplicate lines.
To find the duplicate lines from file, use the below given command
sort file-name|uniq -c -d
In above command :
1. sort – sort lines of text files
2.file-name – Give your file name
3. uniq – report or omit repeated lines
With uniq command, we are using below given parameters :
-c, –count prefix lines by the number of occurrences
-d, –repeated only print duplicate lines
Given below is example
Here, we are find the duplicate lines in file name called list. With cat command, we have shown the content of file.
sharad@linuxworld:/tmp$ cat list Air water soil chemical seas red green flower seas towers trees languages flower boat water sharad@linuxworld:/tmp$ sharad@linuxworld:/tmp$ sort list |uniq -c -d 2 flower 2 seas 2 water sharad@linuxworld:/tmp$