How do I set up the append option for updating data correctly?

sarahi6 · 2023-05-18T19:54:22+00:00

There was an error rendering this rich post.

Use case: I have callrail data that begins in 1/1/2021. Currently the dataset is updating everyday and has REPLACE as an update method. The dataset is now taking at least 1-3 hours to update. I thought about changing the update method to append but when I did, I had duplicate data from 2021.

How can I change the update method to APPEND without get duplicate data? I want the dataset to update every night and append yesterday's data to the dataset.

Find more posts tagged with

Date

Append

Dataset

Domo University

Accepted answers

MichelleH

@sarahi6 Try changing it to a Single Date with a relative 1 day back. If you're setting the dataset to Append then you don't need a static start date

All comments

MichelleH

@sarahi6 You will also have to filter the source data to yesterday's date so you are not duplicating rows. What kind of connector are you using?

sarahi6

The connector is CallRail @MichelleH. Above is the duration set up. Currently I have it updating through yesterday's date.

MichelleH

@sarahi6 Try changing it to a Single Date with a relative 1 day back. If you're setting the dataset to Append then you don't need a static start date