Remove Duplicates
Remove duplicate rows
What It Does
Remove Duplicates keeps only unique rows. Optionally specify columns to determine uniqueness (keeping the first occurrence).
How to Use It
Opening the Panel
Click the + button in the toolbar or press the operation picker, then search for "Remove Duplicates". It's in the Filter & Sort group.
Configuration
| Field | Type | Required | Description |
|---|---|---|---|
Columns | Multi-select | No | Columns to check for duplicates (all columns if empty) |
SQL Generated
SELECT DISTINCT * FROM data
-- Or by specific columns:
SELECT DISTINCT ON (email) * FROM dataNotes
- Null values are handled gracefully - operations skip or preserve nulls where appropriate.
- You can edit any pipeline step by clicking on its card to re-open the panel with the same settings.
- The SQL preview shows exactly what will execute - inspect it before applying.
Related Operations
- Filter - Show only rows matching conditions
- Limit Rows - Keep only the first N rows
- Top / Bottom Rows - Show top or bottom N rows by column value