Can I schedule a date flow to run at a specified time?

Jim_Medina

I created a data flow (thanks to answers from the Community!) which seems to take quite a long time to run manually.  I've submitted it to run twice manually and cancelled it after 30 minutes.  I haven't run it successfully yet, so I don't even know if it works.  If it fails, at least I'll get to find my errors.

 

I'd like to schedule it to run sometime in the early morning before I get in.  Is there a way to do that?

quinnj

Hey @Jim_Medina,

 

I like to use a simple trick to accomplish this using the NOAA Weather connector. As you know, DataFlows can be set to run when any of their datasources update. An easy way to "schedule" a DataFlow to run at a specific time is to setup a "dummy" datasource that will update at the time you'd like the DataFlow to run.

 

I like to use the NOAA Weather connector for this since it only requires a zip code to setup and we can leverage the connector scheduling abilities to essentially "schedule" our DataFlow.

 

To setup the NOAA Weather connector, head to your Data Center and click the "New DataSet" button in the top right. Scroll down and select the NOAA Weather connector (not the weather alerts version). You can then put in any zip code you'd like, click "Next" and then you have the ability to schedule how often you want this connector to update (daily, weekly, every 15 minutes, etc.). Note that the time of day setting is based on UTC time, which will be 5-7 hours ahead of USA timezones.

 

Then you simply include this new "dummy" weather dataset as an input to your DataFlow and scroll down to check the "Run DataFlow when [X] updates" for the weather dataset. You don't even have to do anything with the weather dataset in the actual dataflow, just ignore it; we're just using it for the scheduled update.

 

Hope that helps!