Magic ETL Ideas

Magic ETL Ideas

Conditional Gateway Tile

Greetings,

I would like to propose the addition of a tile that enables early termination of a dataflow without writing to the output dataset when specific conditions are met.

A key use case for this feature would be terminating the dataflow if the input dataset contains zero rows. In such cases, the process should end, and a notification should be sent indicating a failure. This functionality would play a crucial role in our data sanity checks, ensuring data integrity before proceeding further.

Tagged:
7
7 votes

Welcome!

It looks like you're new here. Members get access to exclusive content, events, rewards, and more. Sign in or register to get started.
Sign In

Comments

  • Contributor

    In the past we would setup convoluted triggers where once

    Datasets α and β proccessed, Dataflow Γ would run and give us dataset δ and then we would run a logic check on δ to define ε as a trigger for dataflow ζ. Reducing that into a single tile in a Dataflow would be, dare I say it, Magic?

  • A team member of mine reminded me that you can do this using the error function in Magic ETL

    If I solved your problem, please select "yes" above

  • Could you provide a little more info on this. I am not familiar with what you are referring to.

  • edited March 28

    https://community-forums.domo.com/main/discussion/55570/is-there-a-reliable-way-to-make-magic-etl-intentionally-fail

    If I solved your problem, please select "yes" above

Welcome!

It looks like you're new here. Members get access to exclusive content, events, rewards, and more. Sign in or register to get started.
Sign In