Deduplication
pandas Deduplication Interview Questions
Deduplication is getting back to one row per thing when the source has several, and deciding on purpose which one to keep. Duplicate rows are the most common defect in real data.
What deduplication questions ask you to do
Across the questions here, these are the shapes that come up:
- Keep the most recent row per customer or account
- Remove rows that are identical in every column
- Handle near-duplicates that differ only by a timestamp
- Deduplicate before a join so totals do not inflate
- Count the duplicates before removing them
Common mistakes with deduplication
Removing rows that match on every column leaves behind the near-identical ones that differ in a single timestamp, which is not deduplication at all. Name the column that identifies the thing, name the order that decides which row wins, and then keep the first.
Questions to practice
Questions last reviewed 19 October 2026.