Long running ETL flows alerting strategies
Any strategies i can employ to alert myself whenever a query that generally takes seconds to run runs for longer than it should?
I have a query that has on rare occasions been found running for hours when it should run in seconds. I'd like a way to be alerted to this situation without having to come across it by chance.
Thanks!
Comments
-
you could use the domo_governance dataset or the DomoStats_DataflowHistory (i believe) dataset to see how long it's taking dataflows to execute, but they would have to be updated periodically ... usually Domo prefers the governance datasets to only update daily.
are you running this ETL multiple times a day?
you could build a series of scripts against the dataflow oriented APIs by leveraging the domo command line utility. https://domohelp.domo.com/hc/en-us/articles/360043437733-Command-Line-Interface-CLI-Tool#:~:text=Domo's%20Command%20Line%20Interface%20(CLI,here%20to%20view%20the%20changelog.
alternatively, instead of testing the dataflow, test the artificat, the datasets. If you're updating the every 15 minutes, then just report any dataset that hasn't updated in the last half hour.
Jae Wilson
Check out my 🎥 Domo Training YouTube Channel 👨💻
**Say "Thanks" by clicking the ❤️ in the post that helped you.
**Please mark the post that solves your problem by clicking on "Accept as Solution"0
Categories
- All Categories
- 1.7K Product Ideas
- 1.7K Ideas Exchange
- 1.5K Connect
- 1.2K Connectors
- 292 Workbench
- 4 Cloud Amplifier
- 8 Federated
- 2.8K Transform
- 95 SQL DataFlows
- 600 Datasets
- 2.1K Magic ETL
- 3.7K Visualize
- 2.4K Charting
- 685 Beast Mode
- 43 App Studio
- 38 Variables
- 655 Automate
- 170 Apps
- 438 APIs & Domo Developer
- 42 Workflows
- 5 DomoAI
- 32 Predict
- 12 Jupyter Workspaces
- 20 R & Python Tiles
- 383 Distribute
- 110 Domo Everywhere
- 267 Scheduled Reports
- 6 Software Integrations
- 111 Manage
- 108 Governance & Security
- 8 Domo University
- 25 Product Releases
- Community Forums
- 39 Getting Started
- 29 Community Member Introductions
- 98 Community Announcements
- Domo Community Gallery
- 4.8K Archive