Skip to content
Book a demo

Connect documentation

All documentation
Documentation

Managing Data Quality issues

On this page

This guide explains how to work with the issues that Data Quality monitors find in your data through the Connect API. You can do the following:

  • List and filter monitor issues, read the details of one issue, and view issue analytics.

  • Acknowledge and reopen issues, mark an issue as expected, and accept a changed column structure into the baseline.

  • Check the rules of a custom monitor against a data extract before you save the monitor.

This guide covers what is specific to Data Quality issues: their details, filters, analytics, and actions. The legacy issue list described in Listing issues returns all issues of your datastreams, including Data Quality issues, under the same ID.

The v1 endpoints follow their own conventions for authentication, identifiers, pagination, filters, and errors. For more information, see Working with v1 endpoints.

Introduction

What is a Data Quality issue?

An issue records that a monitor found a problem in a data extract of a datastream, for example duplicate rows, an unusual row count, a late fetch, a changed column structure, or values that break a rule of a custom monitor. For more information, see Viewing details of the Data Quality issues.

How are issues identified?

Each issue has a numeric id: the ID of the log entry that recorded the issue, the same ID that GET /api/errors/ shows. datastreams, monitors, and workspaces are identified by UUID, and data extracts and job groups by numeric ID.

Which issue types exist?

The type field of each issue names the monitor that found it:

Scroll horizontally to see the full table.

type

Monitor

row-duplicates

Duplication monitor

volumes

Volume monitor

timeliness

Timeliness monitor

schema_consistency

Column consistency monitor

values

Custom monitor

Prerequisites

Before you complete the procedures in this guide, perform the following action:

  • Use a Connect API key with the following permissions:

    • Monitor: Read-only (monitor:read) to list and read issues and to view issue analytics.

    • Monitor: Write (monitor:write) to acknowledge, reopen, and mark issues as expected, and to accept a changed column structure into the baseline.

    • Monitor: Read-only (monitor:read) and Datastream: Read-only (datastream:read) to check custom monitor rules against a data extract. The check reveals the values of the data extract, so it needs both permissions.

    Picking Write for a permission also gives the key its Read-only access. Without the permission a request needs, Connect API returns HTTP 403. You cannot change the permissions of a key, so to add one, create a new key. For more information, see Creating a Connect API key in Adverity.

Limitations

  • A request that acknowledges or reopens issues can name at most 50 issue IDs. If it would change more than 1000 issues, Connect API returns HTTP 400 and changes nothing. Narrow the selection with issue IDs, a workspace selection, or a job group.

  • The details of an issue summarize the violations but do not contain the violating or duplicated rows themselves. To view these rows, use the Data Quality page in Adverity.

  • Issues in workspaces that are scheduled for deletion are not listed and cannot be changed.

Selecting workspaces

By default, the requests in this guide cover every issue the API key can see: the issues of the key’s workspace and the workspaces below it. To narrow a request to one part of your workspace tree, add the following query parameters:

Scroll horizontally to see the full table.

Parameter

Description

workspace_scope

ancestors, descendants, or family: only the issues in the workspace given by scope_from_workspace_id and the workspaces above it, below it, or both. The result never includes workspaces the key cannot see.

scope_from_workspace_id

The workspace UUID that workspace_scope starts from. Required when the instance has more than one top-level workspace; otherwise it defaults to the top-level workspace. Without workspace_scope, this parameter has no effect and the request covers every workspace the key can see.

For example, workspace_scope=descendants&scope_from_workspace_id={{WORKSPACE_ID}} selects the issues of one workspace and the workspaces below it, as the Data Quality page of this workspace shows them.

These parameters work on the issue list, the issue analytics, and the requests that acknowledge or reopen issues by ID or by workspace. The requests for the issues of a job group do not accept them.

Listing issues

To list Data Quality issues, follow these steps:

  1. Create a GET request to the following endpoint:

    https://{{INSTANCE}}/api/v1/monitors/issues/
    
  2. In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.

  3. (Optional) To filter the results, add any of the following query parameters. If you combine several parameters, the response contains only the issues that match all of them.

    Scroll horizontally to see the full table.

    Parameter

    Description

    workspace_scope, scope_from_workspace_id

    Only the issues of the selected workspaces. For more information, see Selecting workspaces.

    datastream_id

    Only the issues of these datastream UUIDs. Repeat the parameter for several values, up to 50.

    monitor_id

    Only the issues of these custom monitor UUIDs. Repeat the parameter for several values, up to 50.

    type

    Only the issues of one type: row-duplicates, values, volumes, timeliness, or schema_consistency.

    severity

    error or warning.

    is_acknowledged

    true for acknowledged issues only, false for unacknowledged issues only.

    date_from

    Only the issues whose fetch started on this day or later, as YYYY-MM-DD (UTC).

    date_to

    Only the issues whose fetch started on this day or earlier, as YYYY-MM-DD (UTC). date_from after date_to returns HTTP 400.

    page

    The page number, starting at 1. Default: 1.

    page_size

    The number of items per page. Default: 50. Values above 100 return 100 items.

  4. Send the request.

As a result, you obtain the issues, newest fetch first.

The cURL request example is the following. It lists the unacknowledged errors of one workspace and the workspaces below it:

curl --location --request GET 'https://{{INSTANCE}}/api/v1/monitors/issues/?workspace_scope=descendants&scope_from_workspace_id={{WORKSPACE_ID}}&severity=error&is_acknowledged=false' \
--header 'Authorization: Bearer {{KEY}}'

Example of JSON response (cropped to show one issue):

{
  "items": [
    {
      "id": 81234,
      "type": "values",
      "detailed_type": "rule-violated",
      "severity": "error",
      "start": "2026-10-28T03:00:12Z",
      "is_acknowledged": false,
      "expected": null,
      "violations": 12,
      "message": "{{ISSUE_MESSAGE}}",
      "data_source": "{{CONNECTOR_NAME}}",
      "job_group_id": 5120,
      "datastream": {
        "id": "{{DATASTREAM_UUID}}",
        "name": "{{DATASTREAM_NAME}}",
        "workspace": {
          "id": "{{WORKSPACE_UUID}}",
          "name": "{{WORKSPACE_NAME}}"
        }
      },
      "monitor": {
        "id": "{{MONITOR_UUID}}",
        "name": "{{MONITOR_NAME}}"
      },
      "extract_id": 51234
    }
  ],
  "count": 1
}

The following table describes the fields of each issue:

Scroll horizontally to see the full table.

Field

Description

id

The numeric ID of the issue.

type

The type of the issue. The fields after the common ones depend on the type. For more information, see Reading the details of an issue.

severity

error or warning.

start

When the fetch that raised the issue started.

is_acknowledged

true if the issue is acknowledged.

expected

Whether the issue was marked as expected. null until someone marks it. Only the issues of universal monitors use this field.

violations

The number of violations the monitor found.

message

The issue message.

data_source

The name of the datastream’s connector, as Adverity displays it. This is a display name, not an identifier: it can change and is not unique.

job_group_id

The job group whose fetch raised the issue. null if the issue is not tied to a job group. For a timeliness or column consistency issue, this is the job group the issue recorded itself, which can differ from the job of the issue’s log entry. The job group requests select issues by the job of their log entry, so they can leave out such an issue, for example a timeliness issue recorded without a fetch job. To change such an issue, use its ID.

datastream

The id, name, and workspace of the datastream.

monitor

The id and name of the custom monitor that raised the issue. null for universal monitors.

extract_id

The data extract the issue was found on. null if the data extract was deleted.

Each type adds the following fields:

Scroll horizontally to see the full table.

type

Additional fields

values

detailed_type: missing-field or rule-violated.

row-duplicates

None.

volumes

anomaly_type (spike, drop, or drop_to_zero), deviation, and row_count.

timeliness

expected_delivery, schedule, schedule_type, smart_schedule, and cause (stopped_by_error or task_not_finished).

schema_consistency

issue_types (missing_columns, new_columns, or order_mismatch) and auto_restored.

Reading the details of an issue

To read the details of one issue, follow these steps:

  1. Create a GET request to the following endpoint:

    https://{{INSTANCE}}/api/v1/monitors/issues/{{ISSUE_ID}}/
    

    Replace {{ISSUE_ID}} with the id of the issue.

  2. In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.

  3. Send the request.

As a result, you obtain the issue with the fields described in Listing issues and the following details of its type:

Scroll horizontally to see the full table.

type

Details

values

names: the fields missing from the data extract, for a missing-field issue. rule_violations: for a rule-violated issue, one summary per violated rule block, with the block’s rule_key, the number of violations, the number of distinct_rows recorded up to the row limit, has_target_columns, and rows_truncated and columns_truncated, which are true if the monitor found more rows or columns for that block than it recorded.

row-duplicates

columns and columns_mode: the columns the monitor compared. duplicated_rows: the number of distinct duplicated rows recorded, up to the row limit. rows_truncated and columns_truncated are true if the monitor found more rows or columns than it recorded.

volumes

datapoints: the last 50 fetches the monitor compared, each with its volume, the fetch range, the expected range lower_bound to upper_bound, and is_outlier.

timeliness

due_date: when the data was due. extraction_log_message: the message of the log entry that delayed the fetch. null if there is none or the key cannot see it.

schema_consistency

extract_schema: the columns of the data extract. baseline_schema: the columns of the baseline. diff: the differences between them. issue_types_count: the number of missing columns, new columns, and columns in a different order.

If the issue does not exist or the key cannot see it, Connect API returns HTTP 404.

The cURL request example is the following:

curl --location --request GET 'https://{{INSTANCE}}/api/v1/monitors/issues/{{ISSUE_ID}}/' \
--header 'Authorization: Bearer {{KEY}}'

Listing the issues of a data extract or a job group

To list the issues found on one data extract, or raised by the fetch of one job group, follow these steps:

  1. Create a GET request to one of the following endpoints:

    • For a data extract:

      https://{{INSTANCE}}/api/v1/monitors/issues/extracts/{{EXTRACT_ID}}/
      

      Replace {{EXTRACT_ID}} with the numeric ID of the data extract, for example the extract_id of an issue.

    • For a job group:

      https://{{INSTANCE}}/api/v1/monitors/issues/job-groups/{{JOB_GROUP_ID}}/
      

      Replace {{JOB_GROUP_ID}} with the numeric ID of the job group, for example the job_group_id of an issue. A timeliness or column consistency issue can carry a job_group_id and still not be listed here. For more information, see the job_group_id field in Listing issues.

  2. In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.

  3. Send the request.

As a result, you obtain the issues of the data extract or the job group, paginated with the page and page_size query parameters. Each issue has the fields described in Listing issues.

For a data extract, the response also contains the issues found on copies of the same source data extract that the key can see.

If the data extract does not exist or the key cannot see it, Connect API returns HTTP 404. A job group that does not exist or that the key cannot see returns an empty list.

The cURL request example is the following:

curl --location --request GET 'https://{{INSTANCE}}/api/v1/monitors/issues/job-groups/{{JOB_GROUP_ID}}/' \
--header 'Authorization: Bearer {{KEY}}'

Viewing issue analytics

For details, see Viewing issue analytics and previewing monitor rules.

Acknowledging and reopening issues

For details, see Acknowledging and reopening Data Quality issues.

Marking an issue as expected

For details, see Acknowledging and reopening Data Quality issues.

Accepting a column structure change into the baseline

For details, see Acknowledging and reopening Data Quality issues.

Previewing custom monitor rules on a data extract

For details, see Viewing issue analytics and previewing monitor rules.

What’s next?