When do we use Magic ETL and SQL data flows?
Comments
-
In my experience, anytime I'm doing something with large datasets or process intensive transformations I'll try to do it in the ETL module. ETLs normally run faster than SQL transforms. But there are some things that become cumbersome to 'code' in an ETL transform and then I'll just write what I need in SQL.
But if possible, I try to use ETL.
1 -
I think it depends on the setup and if you have a existing Enterprise Data Warehouse or if you are using DOMO as one. If you do you can try and perform the complex data transformations (column manipulation, generating self joins for rankings, etc...) outside of DOMO. Then push that data in and just use Magic ETL for the remaining lighter transformations.
My organization does not have a EDW so we use DOMO to merge a lot of our datasets from mutliple sources. Seems to me that once the transform get a little complex (which happens often), we need to make the switch to SQL. There is a significant difference in speed though, and I agree 100% that you should try and stick with Magic ETL as much as possible.
1
Categories
- All Categories
- 1.8K Product Ideas
- 1.8K Ideas Exchange
- 1.5K Connect
- 1.2K Connectors
- 300 Workbench
- 6 Cloud Amplifier
- 8 Federated
- 2.9K Transform
- 100 SQL DataFlows
- 616 Datasets
- 2.2K Magic ETL
- 3.9K Visualize
- 2.5K Charting
- 738 Beast Mode
- 56 App Studio
- 40 Variables
- 684 Automate
- 176 Apps
- 452 APIs & Domo Developer
- 46 Workflows
- 10 DomoAI
- 36 Predict
- 15 Jupyter Workspaces
- 21 R & Python Tiles
- 394 Distribute
- 113 Domo Everywhere
- 275 Scheduled Reports
- 6 Software Integrations
- 123 Manage
- 120 Governance & Security
- 8 Domo Community Gallery
- 38 Product Releases
- 10 Domo University
- 5.4K Community Forums
- 40 Getting Started
- 30 Community Member Introductions
- 108 Community Announcements
- 4.8K Archive