Published Oct 04 2020
Psssst! The 2023 WEB DEVELOPMENT BOOTCAMP is starting on FEBRUARY 01, 2023! SIGNUPS ARE NOW OPEN to this 10-weeks cohort course. Learn the fundamentals, HTML, CSS, JS, Tailwind, React, Next.js and much more! ✨
uniq is a command useful to sort lines of text.
You can get those lines from a file, or using pipes from the output of another command:
uniq dogs.txt ls | uniq
You need to consider this key thing:
uniq will only detect adjacent duplicate lines.
This implies that you will most likely use it along with
sort dogs.txt | uniq
sort command has its own way to remove duplicates with the
-u (unique) option. But
uniq has more power.
By default it removes duplicate lines:
You can tell it to only display duplicate lines, for example, with the
sort dogs.txt | uniq -d
You can use the
-u option to only display non-duplicate lines:
You can count the occurrences of each line with the
Use the special combination:
sort dogs.txt | uniq -c | sort -nr
to then sort those lines by most frequent:
uniq command works on Linux, macOS, WSL, and anywhere you have a UNIX environment