FOR WHILE loops in dataflows
Related to: https://dojo.domo.com/main/discussion/52378/is-it-possible-to-use-while-do-loops-in-mysql-in-domo-dataflows
There is no way to implement SQL Recursive Queries in Domo: https://www.sqlservertutorial.net/sql-server-basics/sql-server-recursive-cte/
Instead you have to guess the max depth and build as many JOINs and columns manually into your Queries / Dataflows.
----
It would be ideal to have a mechanism for executing a FOR / WHILE loop in dataflows.
Use cases for my organization include, we have a writeback connector that due to the vendor's API restraints can only receive 1M rows at a time.
If we knew our dataflow could have up to 7M rows, we'd have to implement 7 writeback connectors in the dataflow each with a LIMIT / OFFSET function built in.
Ideal state would be to do a FOR WHILE loop that would handle pagination.
---
If you were doing any sort of bill of materials reporting and you wanted to structure your data so that you knew "for each recipie which items are part of this recipie" you would have to know the maximum depth of all work-in-progress recipies.
ex. I want to know all the components of a Subaru Forester. I'd need to know the components of an engine. each engine is composed of pistons... and stuff. each piston is composed of ... gaskets and rods. each gasket is made from ... stuff. In order to build a report that captures the full BOM i'd have to know the depth each WIP goes to.
Check out my 🎥 Domo Training YouTube Channel 👨💻
**Say "Thanks" by clicking the ❤️ in the post that helped you.
**Please mark the post that solves your problem by clicking on "Accept as Solution"
Comments
-
Rumor has it that Workflows will have the ability to implement FOR WHILE loops ; however, workflows are not intended to be substitutes for ETL, so again, ideal state would be to be able to implement loop structures into ETL
With CTEs I can't see the need in SQL dataflows yet; but then we all know that CURSORs are not great for huge datasets which is one of the primary use cases for Adrenaline.
MySQL 5.6 does not support CTEs, they were however added to MySQL 8, https://www.mysqltutorial.org/mysql-cte/#:~:text=MySQL%20introduced%20the%20common%20table,the%20statements%20in%20this%20tutorial.
Jae Wilson
Check out my 🎥 Domo Training YouTube Channel 👨💻
**Say "Thanks" by clicking the ❤️ in the post that helped you.
**Please mark the post that solves your problem by clicking on "Accept as Solution"0 -
100% agree with jaeW_at_Onyx. We need loop structures in ETL.
1
Categories
- All Categories
- 1.8K Product Ideas
- 1.8K Ideas Exchange
- 1.6K Connect
- 1.2K Connectors
- 300 Workbench
- 6 Cloud Amplifier
- 9 Federated
- 2.9K Transform
- 102 SQL DataFlows
- 626 Datasets
- 2.2K Magic ETL
- 3.9K Visualize
- 2.5K Charting
- 753 Beast Mode
- 61 App Studio
- 41 Variables
- 692 Automate
- 177 Apps
- 456 APIs & Domo Developer
- 49 Workflows
- 10 DomoAI
- 38 Predict
- 16 Jupyter Workspaces
- 22 R & Python Tiles
- 398 Distribute
- 115 Domo Everywhere
- 276 Scheduled Reports
- 7 Software Integrations
- 130 Manage
- 127 Governance & Security
- 8 Domo Community Gallery
- 38 Product Releases
- 11 Domo University
- 5.4K Community Forums
- 40 Getting Started
- 30 Community Member Introductions
- 110 Community Announcements
- 4.8K Archive