HxHippy

uniq

Report or omit repeated lines

command Text ProcessingFile Content

Overview

Filters adjacent matching lines from INPUT (or standard input), writing to OUTPUT (or standard output). Note: uniq does not detect repeated lines unless they are adjacent. You may need to sort the input first.

Syntax

uniq [OPTION]... [INPUT [OUTPUT]]

Common Options

-c, --count

Prefix lines by the number of occurrences.

-d, --repeated

Only print duplicate lines, one for each group.

-D, --all-repeated[=delimit-method]

Print all duplicate lines (delimit-method can be 'none' (default), 'prepend', or 'separate').

-f N, --skip-fields=N

Avoid comparing the first N fields.

-i, --ignore-case

Ignore differences in case when comparing.

-s N, --skip-chars=N

Avoid comparing the first N characters.

-u, --unique

Only print unique lines.

-w N, --check-chars=N

Compare no more than N characters in lines.

Examples

$ sort data.txt | uniq

Sorts data.txt and then removes adjacent duplicate lines.

$ sort words.txt | uniq -c

Sorts words.txt, removes duplicates, and shows the count of each line.

$ sort items.txt | uniq -d

Sorts items.txt and shows only the lines that were repeated.

$ sort log.txt | uniq -u

Sorts log.txt and shows only the lines that appeared exactly once.

$ sort -k2 users.csv | uniq -f 1 -i

Sorts users.csv by the second field, then finds unique lines ignoring the first field and case.

Related Commands

cat

Concatenate and display file content

grep

Print lines matching a pattern

less

Opposite of more; view file content paginated

head

Output the first part of files

tail

Output the last part of files

awk

Pattern scanning and processing language

uniqueduplicaterepeatlinestextfilefilteruniq