Configuring Data Mapping in Management API

Retrieving all tracked columns in an instance

To retrieve all tracked columns, follow these steps:

1. Create a GET request to the following endpoint:

   https://{{INSTANCE}}/api/columns/
   

2. In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.

3. Send the request.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location -g --request GET 'https://{{INSTANCE}}/api/columns/' \
--header 'Authorization: Bearer {{KEY}}'

Retrieving all tracked columns in a datastream

To retrieve all tracked columns per datastream, follow these steps:

1. Create a GET request to the following endpoint:

   https://{{INSTANCE}}/api/datastreams/{{DATASTREAM_ID}}/columns/
   

2. In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.

3. Send the request.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location -g --request GET 'https://{{INSTANCE}}/api/datastreams/{{DATASTREAM_ID}}/columns/' \
--header 'Authorization: Bearer {{KEY}}'

Retrieving details of a specific column

To retrieve details of a specific column, follow these steps:

1. Create a GET request to the following endpoint:

   https://{{INSTANCE}}/api/columns/{{COLUMN_ID}}/
   

2. In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.

3. Send the request.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location -g --request GET 'https://{{INSTANCE}}/api/columns/{{COLUMN_ID}}' \
--header 'Authorization: Bearer {{KEY}}'

Retrieving all available target columns

To retrieve all available target columns in your instance, follow these steps:

1. Create a GET request to the following endpoint:

   https://{{INSTANCE}}/api/target-columns/
   

2. In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.

3. Send the request.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location -g --request GET 'https://{{INSTANCE}}/api/target-columns/' \
--header 'Authorization: Bearer {{KEY}}'

Searching through available target columns by name

This section uses an example in which you search for the target column called day using the name parameter. To search through the available target columns using the name parameter, follow these steps:

1. Create a GET request to the following endpoint:

   https://{{INSTANCE}}/api/target-columns/?name=day
   

2. In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.

3. Send the request.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location -g --request GET 'https://{{INSTANCE}}/api/target-columns/?name=day' \
--header 'Authorization: Bearer {{KEY}}'

Retrieving available options for target column configuration

To retrieve all available options for a new target column, including the optional parameters, follow the steps below:

1. Create an OPTIONS request to the following endpoint:

   https://{{INSTANCE}}/api/target-columns/
   

2. In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.

3. Send the request.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location -g --request OPTIONS 'https://{{INSTANCE}}/api/target-columns/' \
--header 'Authorization: Bearer {{KEY}}'

Creating a new target column

If you could not find a target column matching the necessary configuration, create a new target column which you can later assign to a column retrieved from a datastream. To create a new target column, follow these steps:

1. Create a POST request to the following endpoint:

   https://{{INSTANCE}}/api/target-columns/
   

2. In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.

3. In the HTTP request header, include the parameter Content-Type with value application/json.

4. In the HTTP request body, include the following required parameters:

   {
       "type": "String",
       "name": "{{TARGET_COLUMN_NAME}}",
       "usage": "dimension"
   }
   

   These are the required parameters. The value of the usage parameter is either dimension or metric.

5. Send the request.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location -g --request POST 'https://{{INSTANCE}}/api/target-columns/' \
--header 'Authorization: Bearer {{KEY}}' \
--header 'Content-Type: application/json' \
--data-raw '{
    "type": "String",
    "name": "{{TARGET_COLUMN_NAME}}",
    "usage": "dimension"
}'

Viewing a target column mapping

To view the list of default schema mappings assigned to a target column, follow the steps below:

1. Create a GET request to the following endpoint:

   https://{{INSTANCE}}/api/target-columns/{{TARGET_COLUMN_ID}}/mappings/
   

2. In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.

3. In the HTTP request header, include the parameter Content-Type with value application/json.

4. Optionally, add query parameters to filter the results:

   Scroll horizontally to see the full table.

   Parameter	Description
   is_key_column	Filter by whether the mapped source column is a key column. Accepted values: true, false.
   source_column_name	Filter by the exact name of the source column.
   source_content_type	Filter by the numeric content type ID of the connector the mapping applies to.

5. Send the request.

As a result, you obtain a paginated list of default schema mappings for the target column.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location -g --request GET 'https://{{INSTANCE}}/api/target-columns/{{TARGET_COLUMN_ID}}/mappings/' \
--header 'Authorization: Bearer {{KEY}}' \
--header 'Content-Type: application/json'

Updating column mapping

In this section, you assign a column to a target column, or in other words, you configure the Data Mapping.

To update column mapping, follow these steps:

1. Create a PATCH request to the following endpoint:

   https://{{INSTANCE}}/api/datastreams/{{DATASTREAM_ID}}/columns/{{COLUMN_ID}}/
   

2. In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.

3. In the HTTP request header, include the parameter Content-Type with value application/json.

4. In the HTTP request body, include the following required parameters:

   {
       "is_key_column":"false",
       "set_default": "false",
       "target_column": {"id": {{TARGET_COLUMN_ID}}}
   }
   

5. Send the request.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location -g --request PATCH 'https://{{INSTANCE}}/api/datastreams/{{DATASTREAM_ID}}/columns/{{COLUMN_ID}}/' \
--header 'Authorization: Bearer {{KEY}}' \
--header 'Content-Type: application/json' \
--data-raw '{
    "is_key_column":"false",
    "set_default": "false",
    "target_column": {"id": {{TARGET_COLUMN_ID}}}
}'

Bulk assigning column mappings

Instead of mapping columns one by one, you can assign multiple source columns to target columns in a single request. This operation replaces all existing mappings for the datastream - any column not included in the request body will be left unmapped.

To bulk assign column mappings, follow these steps:

1. Create a PUT request to the following endpoint:

   https://{{INSTANCE}}/api/datastreams/{{DATASTREAM_ID}}/columns/map/
   

2. In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.

3. In the HTTP request header, include the parameter Content-Type with value application/json.

4. In the HTTP request body, include a list of mapping objects. Each object must contain the following parameters:

   • schema_column - the name of the target column to assign to.

   • source_column - the name of the source column from the datastream to map.

   • is_key_column (optional) - a boolean that marks the source column as a key column. When included, the value is applied to the matched source column. When omitted, the column’s existing is_key_column value is preserved. Source columns not included in the request are also unaffected.

   [
       {
           "schema_column": "{{TARGET_COLUMN_NAME}}",
           "source_column": "{{SOURCE_COLUMN_NAME}}",
           "is_key_column": true
       },
       {
           "schema_column": "{{TARGET_COLUMN_NAME}}",
           "source_column": "{{SOURCE_COLUMN_NAME}}"
       }
   ]
   

5. Send the request.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location -g --request PUT 'https://{{INSTANCE}}/api/datastreams/{{DATASTREAM_ID}}/columns/map/' \
--header 'Authorization: Bearer {{KEY}}' \
--header 'Content-Type: application/json' \
--data-raw '[
    {
        "schema_column": "{{TARGET_COLUMN_NAME}}",
        "source_column": "{{SOURCE_COLUMN_NAME}}",
        "is_key_column": true
    }
]'

Removing column mapping

In this section, you remove a column’s mapping (unmap a column) using Management API.

To remove a column’s mapping (unmap a column), follow these steps:

1. Create a PATCH request to the following endpoint:

   https://{{INSTANCE}}/api/datastreams/{{DATASTREAM_ID}}/columns/{{COLUMN_ID}}/
   

2. In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.

3. In the HTTP request header, include the parameter Content-Type with value application/json.

4. In the HTTP request body, include an empty target_column object:

   {
       "target_column": {}
   }
   

5. Send the request.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location -g --request PATCH 'https://{{INSTANCE}}/api/datastreams/{{DATASTREAM_ID}}/columns/{{COLUMN_ID}}/' \
--header 'Authorization: Bearer {{KEY}}' \
--header 'Content-Type: application/json' \
--data-raw '{
   "target_column": {}
}'

Listing all default schema mappings

To list default schema mappings across all target columns, follow these steps:

1. Create a GET request to the following endpoint:

   https://{{INSTANCE}}/api/default-schema-mappings/
   

2. In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.

3. Optionally, add query parameters to filter the results:

   Scroll horizontally to see the full table.

   Parameter	Description
   is_key_column	Filter by whether the mapped source column is a key column. Accepted values: true, false.
   source_column_name	Filter by the exact name of the source column.
   source_content_type	Filter by the numeric content type ID of the connector the mapping applies to.

4. Send the request.

As a result, you obtain a paginated list of default schema mappings.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location -g --request GET 'https://{{INSTANCE}}/api/default-schema-mappings/' \
--header 'Authorization: Bearer {{KEY}}'

Creating a default schema mapping

To create a default schema mapping, follow these steps:

1. Create a POST request to the following endpoint:

   https://{{INSTANCE}}/api/default-schema-mappings/
   

2. In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.

3. In the HTTP request header, include the parameter Content-Type with value application/json.

4. In the HTTP request body, include the following parameters:

   Scroll horizontally to see the full table.

   Parameter	Required	Description
   source_column_name	Yes	The exact name of the source column to map.
   source_content_type	Yes	The numeric content type ID of the connector this mapping applies to.
   target_column_id	Yes	The ID of the target column to map the source column to.
   is_key_column	No	Whether to mark the mapped source column as a key column. Defaults to false.

5. Send the request.

As a result, the default schema mapping is created and returned in the response.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location -g --request POST 'https://{{INSTANCE}}/api/default-schema-mappings/' \
--header 'Authorization: Bearer {{KEY}}' \
--header 'Content-Type: application/json' \
--data-raw '{
    "source_column_name": "{{SOURCE_COLUMN_NAME}}",
    "source_content_type": {{CONTENT_TYPE_ID}},
    "target_column_id": {{TARGET_COLUMN_ID}}
}'

Updating a default schema mapping

To update a default schema mapping, follow these steps:

1. Create a PATCH request to the following endpoint:

   https://{{INSTANCE}}/api/default-schema-mappings/{{MAPPING_ID}}/
   

2. In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.

3. In the HTTP request header, include the parameter Content-Type with value application/json.

4. In the HTTP request body, include any of the parameters documented in Creating a default schema mapping that you want to change.

5. Send the request.

As a result, the default schema mapping is updated and returned in the response.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location -g --request PATCH 'https://{{INSTANCE}}/api/default-schema-mappings/{{MAPPING_ID}}/' \
--header 'Authorization: Bearer {{KEY}}' \
--header 'Content-Type: application/json' \
--data-raw '{
    "is_key_column": true
}'

This endpoint returns the same 409 response described in Creating a default schema mapping if the update collides with another mapping.

Removing a default schema mapping

To remove a default schema mapping, follow these steps:

1. Create a DELETE request to the following endpoint:

   https://{{INSTANCE}}/api/default-schema-mappings/{{MAPPING_ID}}/
   

2. In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.

3. Send the request.

As a result, the default schema mapping is removed.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location -g --request DELETE 'https://{{INSTANCE}}/api/default-schema-mappings/{{MAPPING_ID}}/' \
--header 'Authorization: Bearer {{KEY}}'