Has anyone tried to create a custom schedule for an ETL job?

I am trying to create a daily snapshot of our datasets regardless of whether the data change or not. The current ETL process I have created can do such, so long as the dataset in which I am pulling data is update. However if the data is never updated, the ETL is never kicked off, which can create a gap in our day to day snapshots. Anyone?

I have been told to create a separate dataset that can help trigger my jobs, but how would i schedule the new dataset to run on a specific time, let alone update the data it would be storing to kick off my ETL job? Anyone?

Find more posts tagged with

Accepted answers

CoreVest

Thank you Godzilla but no. The solution i came up with is to create a datasource that consistently runs at a specific time, once created add that source into my MySQL ETL and select the box that runs the job when that data source is update. I hate using work arounds but seeing as to how we are not implementing workbence, it is my only option.

All comments

Godzilla

i'm assuming you're using the current version of workbench correct? if so, you may have to revert back to an older version of workbench to be able to schedule jobs at specific times. I'm not sure if version 3 allows to to control the times but i know version 2 does. I'm still using version 2 because of this.

CoreVest