separating Output datasets from ETL

I think I made a a mistake of having 1 ETL facilitate some simple transformations for several input datasets. I have the ETL triggered to run whenever any of them update... but now it occurred to me that the ETL is going to keep running every time one of them updates - and I'm afraid it won't pull in the latest updates from one input dataset if the ETL is running (because another input dataset updated just prior) while it has updated.

And I have these output datasets linked to a lot of cards....

Before I redo the ETLs and have to update all the cards with their respective datasets and beastmodes (which will be the biggest headache), is there a way to extract a dataset from an ETL and put it into another?

Tagged:

Answers

  • @NathanDorsch Have you tried modifying the dataflow updated settings so that your updates don't overlap? You could either set it up to run on a schedule or have it set to only run once the last dataset has updated. That could save you the headache of redirecting the ETL outputs.

  • Thanks, how would I schedule it to run when the last dataset is updated each day?

  • @NathanDorsch In the Trigger Frequency settings, it sounds you have it set to "Only when Datasets are updated" and have every input dataset checked. Instead, uncheck the boxes for all the datasets except the one you were concerned about not capturing the updates due to timing.