Magic ETL dataflow not running because of no change in data

cwolman
cwolman Contributor
in Magic ETL

Hello,

 

So, I created a magic ETL dataflow that selects a row from a calendar where the shortdate=Current Date and I scheduled this dataflow to run at 6am daily.  The dataflow has not run once becuase it states the following:

 

Your DataFlow was last triggered on 4/3/2019 6:00 AM but there was no change in the data, so it didn't need to run. This DataFlow last ran on 3/23/2019 3:44 PM.

 

If I manually run this dataflow it will update and contain the correct record.  The data would change if the dataflow would run as scheduled.  What is happening and better yet how can I make this dataflow run on schedule?

 

Thank you,

Chris


-----------------
Chris

Best Answer

  • ST_Superman
    ST_Superman Domo Employee
    Answer ✓

    That should work.  Essentially, Domo added a feature a while back that looks at the input data sets and if none of that data changes, then it won't run the job again.  

     

    You need to have some kind of input data set (even if you don't end up using any of the fields) that will change each day.

Answers

  • ST_Superman
    ST_Superman Domo Employee

    If the data sets are not changing, is your expected output changing?  You might want to try bringing in a data set that changes each day.  (like one of the NOAA weather datasets)  That way you would have a change in the input and the data set would fire.

  • cwolman
    cwolman Contributor

    Typically the calendar table will not change.  If I understand you correctly this is why the dataflow does not fire.  Domo will not run at the scheduled time because the calendar table has not changed even though my output would be different.

     

    Maybe I will create a workbench job that runs daily at 6am that outputs the current date and use that to kick off dataflow?


    -----------------
    Chris
  • ST_Superman
    ST_Superman Domo Employee
    Answer ✓

    That should work.  Essentially, Domo added a feature a while back that looks at the input data sets and if none of that data changes, then it won't run the job again.  

     

    You need to have some kind of input data set (even if you don't end up using any of the fields) that will change each day.

  • rbaeza
    rbaeza Member

    I would like to add something in case anyone else in 2021 is having this issue.

    My issue is that the input DaraSet I have is running after this job that is "not running". I needed to make sure the input DataSet ran first on the scheduler, and this fixed my issue!

Categories