Move output dataset from one ETL to another

jimsteph

I'm writing a new version of the ETL that generates our Employee Directory. The existing output dataset from the existing ETL feeds dozens of other ETLs, hundreds of cards, and quite a few external systems via either the SFTP connector or directly through API calls. When I've got my new version all set I'm going to have to do one of two things: either gut the existing ETL and replace the innards with my new logic — copying and pasting always seems to fail with joins and filters, requiring me to manually add them in — or start moving all of those other ETLs, etc., to the new output dataset. Neither of those really appeals, to be honest.

What I want to do instead is move the existing dataset from the old ETL to the new one once the new ETL has been fully tested. Ideally, when the time comes I'd choose the dataset in the same way you currently choose the input datasets. That would trigger the following actions:

• Domo would verify that the schema of the existing dataset matches or is a subset of the schema of the one it's replacing: if fields need to be deleted that can be done after the swap occurs. It would also let you know the impact of the change, the same way deleting a dataset does.
• It would orphan (and optionally delete) the dataset being replaced, and move the existing one into the new ETL.
• In the original ETL it would replace the dataset being moved with an empty dataset, and optionally either set the ETL to disabled or switch the triggers to manual.

Doing it this way allows you to keep the existing ETL around if needed, or to easily roll everything back in case you didn't catch an issue.

jimsteph

I apologize: I didn't see that @Parker_Lake submitted the same idea a few weeks ago. My search skills seem to be lacking, and I don't think I can delete my original post.