Long running ETL flows alerting strategies
Any strategies i can employ to alert myself whenever a query that generally takes seconds to run runs for longer than it should?
I have a query that has on rare occasions been found running for hours when it should run in seconds. I'd like a way to be alerted to this situation without having to come across it by chance.
Thanks!
Comments
-
you could use the domo_governance dataset or the DomoStats_DataflowHistory (i believe) dataset to see how long it's taking dataflows to execute, but they would have to be updated periodically ... usually Domo prefers the governance datasets to only update daily.
are you running this ETL multiple times a day?
you could build a series of scripts against the dataflow oriented APIs by leveraging the domo command line utility. https://domohelp.domo.com/hc/en-us/articles/360043437733-Command-Line-Interface-CLI-Tool#:~:text=Domo's%20Command%20Line%20Interface%20(CLI,here%20to%20view%20the%20changelog.
alternatively, instead of testing the dataflow, test the artificat, the datasets. If you're updating the every 15 minutes, then just report any dataset that hasn't updated in the last half hour.
Jae Wilson
Check out my 🎥 Domo Training YouTube Channel 👨💻
**Say "Thanks" by clicking the ❤️ in the post that helped you.
**Please mark the post that solves your problem by clicking on "Accept as Solution"0
Categories
- All Categories
- 1.9K Product Ideas
- 1.9K Ideas Exchange
- 1.6K Connect
- 1.3K Connectors
- 303 Workbench
- 6 Cloud Amplifier
- 9 Federated
- 2.9K Transform
- 104 SQL DataFlows
- 640 Datasets
- 2.2K Magic ETL
- 4K Visualize
- 2.5K Charting
- 767 Beast Mode
- 70 App Studio
- 43 Variables
- 716 Automate
- 185 Apps
- 461 APIs & Domo Developer
- 56 Workflows
- 14 DomoAI
- 39 Predict
- 16 Jupyter Workspaces
- 23 R & Python Tiles
- 402 Distribute
- 116 Domo Everywhere
- 277 Scheduled Reports
- 9 Software Integrations
- 134 Manage
- 131 Governance & Security
- 8 Domo Community Gallery
- 44 Product Releases
- 12 Domo University
- 5.4K Community Forums
- 40 Getting Started
- 30 Community Member Introductions
- 111 Community Announcements
- 4.8K Archive