Trigger Frequency Option - When All Datasets Updated

Micah209
Micah209 Member
edited October 2024 in Magic ETL Ideas

I think it would be incredibly useful to add an option in the Dataflow Settings - When Should this Dataflow run section. The option would be 'When all datasets have been updated'. So there would be 4 going forward;

  • Manually
  • On a schedule
  • When selected datasets have been updated
  • When all datasets have been updated

The current suggested method to add the dataflow to a schedule that runs after the last set is updated only works if run times are consistent. I have a lot of datasets that have large variations in run times and I spend most morning reviewing dataflows to ensure they were updated at the right time. I know I'm not the only one that experiences this issue.

Thanks,

Micah

9
Up
9 votes

Active · Last Updated

Comments

  • Micah209
    Micah209 Member

    Looks like I can't edit or delete my post but wanted to correct the post. The category split should be

    • When selected datasets have been updated After each dataset has been updated
    • When all datasets have been updated After all datasets have been updated

    It' would be awesome to have dataflow wait until all selected datasets have run before the it kicks off.

  • jaeW_at_Onyx
    jaeW_at_Onyx Coach

    it's not in product yet, but you can do this with scripting.


    https://www.youtube.com/watch?v=v-8HYp6b4KU

    Jae Wilson
    Check out my 🎥 Domo Training YouTube Channel 👨‍💻

    **Say "Thanks" by clicking the ❤️ in the post that helped you.
    **Please mark the post that solves your problem by clicking on "Accept as Solution"
  • Micah209
    Micah209 Member

    Thank Jae, this is awesome. I think this should be native DOMO functionality but appreciate the workaround.

Categories