• Skip to primary navigation
  • Skip to main content
sharadchhetri

sharadchhetri

Tutorials On Linux, Unix & Open Source

  • Home
  • Linux Commands
  • Resources
    • Learn Linux
  • My WordPress plugins

Unix / Linux : How to print duplicate lines from file

September 12, 2014 by Sharad Chhetri

In this post I am sharing a small tip which is very useful while for system admins. The tip is about how to print duplicate lines from file. Sometimes we get the big file with lot of contents.And there may be chances of duplicate lines.

To find the duplicate lines from file, use the below given command

sort file-name|uniq -c -d 

In above command :

1. sort – sort lines of text files

2.file-name – Give your file name

3. uniq – report or omit repeated lines

With uniq command, we are using below given parameters :
-c, –count prefix lines by the number of occurrences
-d, –repeated only print duplicate lines

Given below is example
Here, we are find the duplicate lines in file name called list. With cat command, we have shown the content of file.

sharad@linuxworld:/tmp$ cat list
Air
water
soil
chemical
seas
red
green
flower
seas
towers
trees
languages
flower
boat
water
sharad@linuxworld:/tmp$ 
sharad@linuxworld:/tmp$ sort list |uniq -c -d
      2 flower
      2 seas
      2 water
sharad@linuxworld:/tmp$ 

Share this:

  • Twitter
  • Facebook
  • More
  • Print
  • Email
  • LinkedIn
  • Reddit
  • Tumblr
  • Pinterest
  • Pocket
  • Telegram
  • WhatsApp
  • Mastodon

Related posts:

  1. print working directory ( pwd , PWD , OLDPWD ) in Linux / Unix
  2. print new line character in Unix / Linux
  3. grep command to remove blank lines from file
  4. awk command to print columns from file
  5. 4 Linux commands to get number of lines in file
  6. grep command to show lines after and before the keyword
  7. grep command to remove commented lines
  8. tr command to convert lines to space , tab and vertical tab
  9. Give permission reference to file/dir from other file/dir in linux
  10. group ownership reference to file/dir from other file/dir in linux

Filed Under: Linux Commands, Tips And Tricks Tagged With: linux command

Copyright © 2023 ยท
The material in this site cannot be republished either online or offline, without our permission.
Proudly Blogging From Bharat.

  • Contact
  • About Me
  • My WordPress plugins
  • Privacy Policy