Append to dataset based on month and year partition value

SivaranjaniMano

Hi everyone,

 

I have a table in my database that is updated everymonth and the table has a partition column like 201911 for Nov 2019, 201912 for Dec 2019 and 201001 for Jan 2020. I want to append these values from the table to a Domo dataset every month without duplicates but not sure how to do that using the replacement variable option.

 

Thanks,

Sivaranjani

Aditya_Jain

Hi Sivaranjani,

If the data you get every month does not have previous months' data then you can follow the steps mentioned below,

1. Go to the dataset settings tab
2. Under the options, go to scheduling,
3. Check for update method, and set it to 'Append' 

Doing this, whenever you refresh the dataset, new rows would append to the existing dataset. But if your data contains past months, then duplication may happen.

 

If your database table is giving out all the months' data then you can go with the 'Replace' option under the same settings as mentioned above and that would clear the domo dataset and will fill it with the incoming data from your database table. 

 

Hope this helps.

 

Regards

Aditya Jain

MarkSnodgrass

If you are not able to use the append method that is available in the Settings section as previously mentioned, you can build a recursive dataflow that would let you append new records to a dataset without duplicating your data. This KB article does a good job explaining it.

https://knowledge.domo.com/Prepare/DataFlow_Tips_and_Tricks/Creating_a_Recursive%2F%2FSnapshot_ETL_DataFlow

Hope this helps.