Skip to content
Book a demo

Connect documentation

All documentation
Documentation

Fetching data

On this page

This guide explains how to fetch data manually, view the collected data, and create and configure scheduled fetches.

Introduction

A fetch is the process of collecting data using a datastream. There are three types of fetches:

  • Manual – start a single fetch manually

  • Smart – intelligently automates data collection by optimizing fetch timing, respecting API rate limits, and applying connector-specific attribution windows

  • Custom – set up regular, automated fetches or schedule a one-time fetch at a specific time, with precise control over schedule and time range

Smart Schedule uses advanced optimization to determine the best times for data collection, minimizing processing errors while ensuring data accuracy through appropriate attribution windows. This eliminates the need to manually calculate and manage complex scheduling parameters.

This video guide explains how to set up and configure scheduled data fetches.

Prerequisites

Before you complete the procedure in this guide, perform all of the following actions:

Fetching data manually

After creating a datastream, fetch data manually to perform a single fetch that will collect data for the date range you choose. This is a single fetch and will not be repeated automatically.To fetch data manually, follow these steps:

  1. Go to the Datastreams page.

  2. Open the chosen datastream by clicking on its name.

  3. In the top right corner, click Fetch now.

  4. In Choose date range, select the time period for which data is collected.

  5. (Optional) Click image1 Advanced options to adjust the following settings:

    Fetch type

    Select one of the following options:

    • Select Run a single fetch including data from start until end to fetch all the data within the selected time period.

    • Select Rerun scheduled fetches in the time period specified to rerun scheduled fetches within the selected time period.

    Keep data in raw state

    Select this checkbox to achieve the following goals:

    • Keep the data in its original form and do not apply any transformation assigned to this datastream. For more information on enriching your data, see Introduction to the Transformations page.

    • Fetch the data without sending it to the destination that you specified for this datastream. For more information on loading data into a destination, see Introduction to the Destinations page.

    Consolidate extract by fetch

    Select this checkbox to fetch the data into one data extract. If unchecked, Adverity creates a different data extract for each day.

    Split into smaller fetches

    Select this checkbox to split a large fetch into multiple smaller fetches in order to avoid errors and improve performance. For example, if you are fetching data for a time period of six months, you can split this fetch into six one-month fetches.

    If you select this checkbox, in the Split by drop-down menu below, select the time period for the smaller fetches. You can choose one of the following options: days, weeks, months or quarters.

  6. (Optional) Define optional filters for this fetch. The filters available in this section depend on the chosen connector.

  7. Click Fetch.

Viewing fetched data

To view the data you collected in your most recent fetch, follow these steps:

  1. Go to the Datastreams page.

  2. Open the chosen datastream by clicking on its name.

  3. In the All tasks tab, find the task at the top of the list, and click on the task overview.

As a result, you can see a table containing the data that you have fetched.

Here, you can perform a range of actions, including applying transformations to your data extract, downloading your collected data, and configuring Data Mapping.

Smart and Custom fetches

You can automate data collection using Smart Schedule or Custom Schedule. Smart Schedule uses intelligent automation to optimize timing, factor in the average duration from previous successful fetches, and handle dependencies, while Custom Schedule allows you to define a precise schedule.

For the steps, see Creating and configuring scheduled fetches and Using Smart Schedule.

Creating a scheduled fetch

For details, see Creating and configuring scheduled fetches.

Using Smart Schedule

For details, see Using Smart Schedule.

Applying Smart Schedule to multiple datastreams

For details, see Using Smart Schedule.

Configuring scheduled fetches

For details, see Creating and configuring scheduled fetches.