• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
sharadchhetri

sharadchhetri

Tutorials On Linux, Unix & Open Source

  • Home
  • Linux Commands
  • Resources
    • Learn Linux
  • My WordPress plugins

Unix / Linux : How to print duplicate lines from file

September 12, 2014 by Sharad Chhetri Leave a Comment

In this post I am sharing a small tip which is very useful while for system admins. The tip is about how to print duplicate lines from file. Sometimes we get the big file with lot of contents.And there may be chances of duplicate lines.

To find the duplicate lines from file, use the below given command

sort file-name|uniq -c -d 

In above command :

1. sort – sort lines of text files

2.file-name – Give your file name

3. uniq – report or omit repeated lines

With uniq command, we are using below given parameters :
-c, –count prefix lines by the number of occurrences
-d, –repeated only print duplicate lines

Given below is example
Here, we are find the duplicate lines in file name called list. With cat command, we have shown the content of file.

sharad@linuxworld:/tmp$ cat list
Air
water
soil
chemical
seas
red
green
flower
seas
towers
trees
languages
flower
boat
water
sharad@linuxworld:/tmp$ 
sharad@linuxworld:/tmp$ sort list |uniq -c -d
      2 flower
      2 seas
      2 water
sharad@linuxworld:/tmp$ 

Share this:

  • Twitter
  • Facebook
  • More
  • Print
  • Email
  • LinkedIn
  • Reddit
  • Tumblr
  • Pinterest
  • Pocket
  • Telegram
  • WhatsApp
  • Mastodon

Related posts:

  1. print working directory ( pwd , PWD , OLDPWD ) in Linux / Unix
  2. print new line character in Unix / Linux
  3. grep command to remove blank lines from file
  4. awk command to print columns from file
  5. 4 Linux commands to get number of lines in file
  6. grep command to show lines after and before the keyword
  7. grep command to remove commented lines
  8. tr command to convert lines to space , tab and vertical tab
  9. Give permission reference to file/dir from other file/dir in linux
  10. group ownership reference to file/dir from other file/dir in linux

Filed Under: Linux Commands, Tips And Tricks Tagged With: linux command

Reader Interactions

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Primary Sidebar

Our Social Media Presence

  • Facebook
  • GitHub
  • Twitter

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Linux Commands

Yum command to download rpm file without installing in linux system

How to set motd in linux

Linux Basic Commands For Every Beginner

man command not found in CentOS and Red Hat

6 df Command Examples To Check Mounted Filesystem On Linux

sed : find the pattern (keyword) and delete the line from file

How to install libxml2 and libxslt packages on CentOS 6.5

Explore 70+ Articles On Linux Commands

Always Useful Tips And Tricks

Error: Could not stat() command file ‘/var/lib/nagios3/rw/nagios.cmd’!

Show visitor ip address php code

configure: error: C++ compiler cannot create executables

SSH WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED

safest method to remove softlink in linux

How to page scroll up/down in Linux terminal

GNU Screen

Explore 90+ Article On "Linux Tips And Tricks"

You Might Like These Articles!

Internal External Command

What is Linux/Unix Internal And External Command

simplecodesyntax wordpress plugin

SimpleCodeSyntax : My Another WordPress Plugin

Install Nginx

How To Install Nginx On Ubuntu 22.04 LTS

Install Latest Git package in Ubuntu Operating System

How To Always Install Latest Git Package In Ubuntu Operating System

Bash script for installing VirtualBox on Ubuntu 22.04 LTS Desktop

Install VirtualBox On Ubuntu 22.04 LTS Desktop (Bash Script)

libfuse

dlopen(): error loading libfuse.so.2 – Got Error On Ubuntu

Copyright © 2023 ยท
The material in this site cannot be republished either online or offline, without our permission but Schools and Colleges can do in their Private Network
Proudly Blogging From Bharat.

  • Contact
  • About Me
  • My WordPress plugins
  • Privacy Policy