Skip to content

Linux commands: uniq

A quick guide to the `uniq` command, used to work with duplicate records/lines in text

uniq is a command useful to sort lines of text.

You can get those lines from a file, or using pipes from the output of another command:

uniq dogs.txt

ls | uniq

You need to consider this key thing: uniq will only detect adjacent duplicate lines.

This implies that you will most likely use it along with sort:

sort dogs.txt | uniq

The sort command has its own way to remove duplicates with the -u (unique) option. But uniq has more power.

By default it removes duplicate lines:

You can tell it to only display duplicate lines, for example, with the -d option:

sort dogs.txt | uniq -d

You can use the -u option to only display non-duplicate lines:

You can count the occurrences of each line with the -c option:

Use the special combination:

sort dogs.txt | uniq -c | sort -nr

to then sort those lines by most frequent:

The uniq command works on Linux, macOS, WSL, and anywhere you have a UNIX environment

β†’ Download my free CLI Handbook!

THE VALLEY OF CODE

THE WEB DEVELOPER's MANUAL

You might be interested in those things I do:

  • Learn to code in THE VALLEY OF CODE, your your web development manual
  • Find a ton of Web Development projects to learn modern tech stacks in practice in THE VALLEY OF CODE PRO
  • I wrote 16 books for beginner software developers, DOWNLOAD THEM NOW
  • Every year I organize a hands-on cohort course coding BOOTCAMP to teach you how to build a complex, modern Web Application in practice (next edition February-March-April-May 2024)
  • Learn how to start a solopreneur business on the Internet with SOLO LAB (next edition in 2024)
  • Find me on X

Related posts that talk about cli: