Option to manually run Magic ETL v2 without optimization
I love that ETL v2 allows automatic query optimization because it significantly cuts down the run time of our ETL pipeline. However I have run into some issues with some rows not being processed when rows get appended to the input dataset while the dataflow is running. The next time I run the dataflow, the ETL optimization thinks that those rows have already been processed because they were added to the input before the dataflow was run last (based on dataflow end time). As a result, I have to temporarily edit the dataflow and opt out of optimization so that I can manually rerun the dataflow with all rows, not just the ones recently added.
It would be helpful to have an option to manually run the dataflow without the optimization so that it's forced to process every row, but continue to optimize the regularly scheduled runs.
Categories
- All Categories
- 1.9K Product Ideas
- 1.9K Ideas Exchange
- 1.6K Connect
- 1.3K Connectors
- 303 Workbench
- 6 Cloud Amplifier
- 9 Federated
- 3K Transform
- 104 SQL DataFlows
- 640 Datasets
- 2.2K Magic ETL
- 4K Visualize
- 2.5K Charting
- 769 Beast Mode
- 72 App Studio
- 43 Variables
- 718 Automate
- 185 Apps
- 462 APIs & Domo Developer
- 57 Workflows
- 14 DomoAI
- 40 Predict
- 17 Jupyter Workspaces
- 23 R & Python Tiles
- 402 Distribute
- 116 Domo Everywhere
- 277 Scheduled Reports
- 9 Software Integrations
- 135 Manage
- 132 Governance & Security
- 8 Domo Community Gallery
- 44 Product Releases
- 12 Domo University
- 5.4K Community Forums
- 40 Getting Started
- 30 Community Member Introductions
- 111 Community Announcements
- 4.8K Archive