Long running ETL flows alerting strategies
Any strategies i can employ to alert myself whenever a query that generally takes seconds to run runs for longer than it should?
I have a query that has on rare occasions been found running for hours when it should run in seconds. I'd like a way to be alerted to this situation without having to come across it by chance.
Thanks!
Comments
-
you could use the domo_governance dataset or the DomoStats_DataflowHistory (i believe) dataset to see how long it's taking dataflows to execute, but they would have to be updated periodically ... usually Domo prefers the governance datasets to only update daily.
are you running this ETL multiple times a day?
you could build a series of scripts against the dataflow oriented APIs by leveraging the domo command line utility. https://domohelp.domo.com/hc/en-us/articles/360043437733-Command-Line-Interface-CLI-Tool#:~:text=Domo's%20Command%20Line%20Interface%20(CLI,here%20to%20view%20the%20changelog.
alternatively, instead of testing the dataflow, test the artificat, the datasets. If you're updating the every 15 minutes, then just report any dataset that hasn't updated in the last half hour.
Jae Wilson
Check out my 🎥 Domo Training YouTube Channel 👨💻
**Say "Thanks" by clicking the ❤️ in the post that helped you.
**Please mark the post that solves your problem by clicking on "Accept as Solution"0
Categories
- 10.5K All Categories
- 8 Connect
- 918 Connectors
- 250 Workbench
- 470 Transform
- 1.7K Magic ETL
- 69 SQL DataFlows
- 477 Datasets
- 193 Visualize
- 252 Beast Mode
- 2.1K Charting
- 11 Variables
- 17 Automate
- 354 APIs & Domo Developer
- 89 Apps
- 3 Workflows
- 20 Predict
- 5 Jupyter Workspaces
- 15 R & Python Tiles
- 247 Distribute
- 63 Domo Everywhere
- 243 Scheduled Reports
- 21 Manage
- 42 Governance & Security
- 174 Product Ideas
- 1.2K Ideas Exchange
- 12 Community Forums
- 27 Getting Started
- 14 Community Member Introductions
- 55 Community News
- 4.5K Archive