Skip to content

Remove Duplicates#

Use the Remove Duplicates node to identify and delete items that are identical across all fields or a subset of fields. This is helpful in situations where you can end up with duplicate data, such as a user creating multiple accounts, or a customer submitting the same order multiple times. When working with large datasets it becomes more difficult to spot and remove these items.

Node parameters#

  • Compare: Select which fields of the input data n8n should compare to check if they're the same. The following options are available:
    • All Fields: Compares all fields of the input data.
    • All Fields Except: Enter which input data fields n8n should exclude from the comparison. You can provide multiple values separated by commas.
    • Selected Fields: Enter which input data fields n8n should include in the comparison. You can provide multiple values separated by commas.

Node options#

If you choose All Fields Except or Selected Fields as your compare type, you can add these options:

  • Disable Dot Notation: Set whether to use dot notation to reference child fields in the format parent.child (turned off) or not (turn on).
  • Remove Other Fields: Set whether to remove any fields that aren't used in the comparison (turned on) or not (turned off).

Templates and examples#

Autonomous AI crawler

by Oskar

View template details
Automate Competitor Research with Exa.ai, Notion and AI Agents

by Jimleuk

View template details
Extract emails from website HTMLs

by Imperol

View template details
Browse Remove Duplicates integration templates, or search all templates

Learn more about data structure and data flow in n8n workflows.