distinct

Introduction

Use the distinct instruction to find the distinct rows in a data extract, and to return how many times the rows appear in the data extract.

To find only duplicate values, use the duplicates transformation.

To find distinct rows and aggregate the metric values, use the aggregate transformation.

When to use this instruction

Use distinct when:

• You want to collapse identical rows into a single row and count how often each combination occurs.

• You need to understand the cardinality of key combinations before choosing keys for joins, aggregations, or uniqueness checks.

If you want to enforce uniqueness and fail when duplicates appear, use verifyunique instead.

Creating a custom script transformation using the distinct instruction

To create and configure a custom script using the distinct instruction, follow these steps:

1. Create a custom script transformation.

2. In the Instruction step, select the distinct instruction.

3. To configure the custom script instruction, fill in the following fields. Required fields are marked with an asterisk (*).

Key

Enter the names of the columns to be used when checking for distinct rows. Click to add columns.

Count

Enter the name of the column that is to be added to the data extract. This column displays the number of distinct rows for each unique combination of values found in the columns specified in Keys.

Presorted

Select this checkbox if the included data extracts are presorted.

Subtable

Enter the name for a subtable that you want to use within this custom script.

A subtable is a temporary table that only exists for this custom script. You can apply additional instructions within the same custom script to the subtable. However, the subtable cannot be used in any other custom scripts.

If a subtable does not exist for the current custom script, the transformation is applied to the data extract, and the enriched data is output into the subtable. If the subtable already exists for the custom script, the subtable is used as the input for the transformation and optionally as the output.

Example