I am trying to create a daily snapshot of our datasets regardless of whether the data change or not. The current ETL process I have created can do such, so long as the dataset in which I am pulling data is update. However if the data is never updated, the ETL is never kicked off, which can create a gap in our day to day snapshots. Anyone?
I have been told to create a separate dataset that can help trigger my jobs, but how would i schedule the new dataset to run on a specific time, let alone update the data it would be storing to kick off my ETL job? Anyone?