Dataflow Run based on All input Dataset refresh Time

Aditya_Jain
Aditya_Jain Contributor
edited February 17 in Ideas Exchange

We are facing a big issue. Let's say I have 2 data sets 'A' and 'B' and these are used as input datasets in a ETL flow. I have set the dataflow setting as - 'Run each time Data set 'A' or 'B' refreshes'.
Problem statement - 

Let's say Dataset A got refreshed before Dataset B, so the ETL/Dataflow would start running as soon as Dataset A got refershed. Meanwhile, the Dataset B also got refreshed, but since the dataflow is already running, those changes in dataset B would not be applied to the final output Dataset.

This problem becomes even bigger  where we have multiple source as inputs.

'Happy to Help'
Tagged:
2
2 votes

· Last Updated

This discussion has been closed.