How do I set up the append option for updating data correctly?

sarahi6
sarahi6 Member
in Connectors

Use case: I have callrail data that begins in 1/1/2021. Currently the dataset is updating everyday and has REPLACE as an update method. The dataset is now taking at least 1-3 hours to update. I thought about changing the update method to append but when I did, I had duplicate data from 2021.

How can I change the update method to APPEND without get duplicate data? I want the dataset to update every night and append yesterday's data to the dataset.

Tagged:

Best Answer

  • MichelleH
    MichelleH Coach
    Answer ✓

    @sarahi6 Try changing it to a Single Date with a relative 1 day back. If you're setting the dataset to Append then you don't need a static start date

Answers

  • MichelleH
    MichelleH Coach

    @sarahi6 You will also have to filter the source data to yesterday's date so you are not duplicating rows. What kind of connector are you using?

  • sarahi6
    sarahi6 Member
    image.png

    The connector is CallRail @MichelleH. Above is the duration set up. Currently I have it updating through yesterday's date.

  • MichelleH
    MichelleH Coach
    Answer ✓

    @sarahi6 Try changing it to a Single Date with a relative 1 day back. If you're setting the dataset to Append then you don't need a static start date

Categories