Recursive dataflow monthly refresh

Jeanvion

Hi,

 

We had a specific requirement to build a recursive dataflow where the incremental data would comprise 2 seperate datasets BUT they want to replace the output once a month with the historical dataset to make sure the data is complete. This requirement is for about 30 dataflows hence they want to reduce the number of steps in maintenance as much as possible. In total we have 4 datasets in our MySQL transformation:

 

• Historical data (which they want to upload once a month)
• Append or replace data
• Delete data
• Output data (which is used as an import dataset to append to)

We provided a solution by creating a date flag in the historical data. If upload date column from historical is > 24 hours from now then replace output dataset with historical data. If not then use normal transformation to append to the output data. Although this solutions works just fine I think it could be useful for Domo to create an import parameter on which dataset is refreshed which could be used in the transformation logic. Simular as the checkboxes you have when to run the dataflow automatically in the devtools. 

 

{
"dataSourceId":"a0e6a8b3-90ba-4d10-ab6a-38421f3c67e1",
"executeFlowWhenUpdated":false,
"dataSourceName":"xxxx"
}],

 

Or alternatively allow to use the output dataset as an import dataset in a seperate dataflow.

 

Any ideas if this is currently possible or might be in the future for simular requirements?

 

@davidf

BCorless

@Jeanvion - This is not currently something avaialble, but I can definitely add this to our suggestion list with development to determine if it is possible. This was created under a Domo Support case: 01906941.

Jeanvion

@macrobolic Thanks for confirming and submitting it as a feature request!