How to use this tool
- Paste one entry per line and choose the lines to keep.
- Set edge trimming, case matching and empty-line handling; review the updated result in first-occurrence order.
- Copy or download the list, or use output as input. Undo last replacement restores the text before a replacement or Clear text.
Understanding the output
Each matching group retains its first line after optional edge trimming. One copy of every line includes all groups; Only lines that repeat includes groups with at least two entries; Only lines appearing once includes groups with one entry. All modes keep first-occurrence order and first capitalization. Unique lines counts the output entries. Duplicates removed counts extra occurrences across all input groups, before the selection filter; Repeated groups counts those with at least two entries. Blank lines skipped are counted separately.
To arrange the unique entries afterward, use Sort Lines with the sorting order that matches your list.
Three lines become two entries
The list apple, pear, apple on separate lines becomes apple followed by pear. The output keeps the first apple rather than sorting the items alphabetically.
Decide what makes two entries equal
A repeated label can look identical while containing a leading space or different capitalization. Turning on trimming or case-insensitive matching can resolve these inconsistencies, but those choices are not always appropriate for identifiers. A case-sensitive code, file path, or password may distinguish values that ordinary prose would treat as equal.
Start with a small sample containing the edge cases you care about. Compare the retained entries with the source before replacing a larger list. This is especially important when whitespace is meaningful, such as indented data or fixed-width text.
Deduplication is different from validation
Removing repeated email addresses, product labels, or URLs does not prove that any remaining entry is valid or active. The tool compares lines, not the underlying people or resources they might represent. Different spellings of the same person remain different entries, and one shared address used by several people becomes a single line when it matches exactly.
Find repeated entries before discarding them
Only lines that repeat shows one representative per repeated group, which helps inspect duplicated labels or identifiers. Only lines appearing once hides every repeated group completely. Neither option sorts the result or validates an address, URL or person represented by a line.
Method and supported input
A map stores each comparison key, its first visible line and occurrence count. Optional trimming and lowercase matching run before grouping. The selected occurrence policy filters complete groups without changing first-occurrence order.
- Each logical record occupies one line.
- The selected case and whitespace policy applies to every record.
Limitations
- No fuzzy matching, spelling correction, or identity matching is performed.
- Unicode characters that look alike can remain distinct.
Common questions
Will this sort the remaining list?
No. It keeps original first-occurrence order. Use the line sorter for alphabetical or natural order.
Does ignoring case change the visible capitalization?
No. The first matching line keeps its capitalization, although optional edge trimming still applies.
Can I find duplicates without including every unique entry?
Yes. Select Only lines that repeat. Each repeated group appears once, using its first line and capitalization.
Sources and further reading
Reviewed on