Long running ETL flows alerting strategies
Any strategies i can employ to alert myself whenever a query that generally takes seconds to run runs for longer than it should?
I have a query that has on rare occasions been found running for hours when it should run in seconds. I'd like a way to be alerted to this situation without having to come across it by chance.
Thanks!
Comments
-
you could use the domo_governance dataset or the DomoStats_DataflowHistory (i believe) dataset to see how long it's taking dataflows to execute, but they would have to be updated periodically ... usually Domo prefers the governance datasets to only update daily.
are you running this ETL multiple times a day?
you could build a series of scripts against the dataflow oriented APIs by leveraging the domo command line utility. https://domohelp.domo.com/hc/en-us/articles/360043437733-Command-Line-Interface-CLI-Tool#:~:text=Domo's%20Command%20Line%20Interface%20(CLI,here%20to%20view%20the%20changelog.
alternatively, instead of testing the dataflow, test the artificat, the datasets. If you're updating the every 15 minutes, then just report any dataset that hasn't updated in the last half hour.
Jae Wilson
Check out my 🎥 Domo Training YouTube Channel 👨💻
**Say "Thanks" by clicking the ❤️ in the post that helped you.
**Please mark the post that solves your problem by clicking on "Accept as Solution"0
Categories
- All Categories
- 2K Product Ideas
- 2K Ideas Exchange
- 1.6K Connect
- 1.3K Connectors
- 311 Workbench
- 6 Cloud Amplifier
- 9 Federated
- 3.8K Transform
- 656 Datasets
- 115 SQL DataFlows
- 2.2K Magic ETL
- 811 Beast Mode
- 3.3K Visualize
- 2.5K Charting
- 80 App Studio
- 45 Variables
- 771 Automate
- 190 Apps
- 481 APIs & Domo Developer
- 77 Workflows
- 23 Code Engine
- 36 AI and Machine Learning
- 19 AI Chat
- AI Playground
- AI Projects and Models
- 17 Jupyter Workspaces
- 410 Distribute
- 120 Domo Everywhere
- 280 Scheduled Reports
- 10 Software Integrations
- 142 Manage
- 138 Governance & Security
- 8 Domo Community Gallery
- 48 Product Releases
- 12 Domo University
- 5.4K Community Forums
- 41 Getting Started
- 31 Community Member Introductions
- 114 Community Announcements
- 4.8K Archive