Can you use a Datafusion dataset in Magic ETL as an Input data source?
![MReznick](https://us.v-cdn.net/6032830/uploads/lithium_avatars/n1920.jpg)
I am trying to join two tables and they will not produce any results in the Magic ETL, But the join works in the Datafusion.
Can we use the Datafusion as a Data input in the Magic ETL?
Best Answer
-
Currently it's not possible to have a DataFusion dataset as an input to Magic ETL. The reasoning, without getting to technical, is that a DataFusion is essentially only creating a view or "description" of how datasets should be joined, without actually running any calculations or physically creating a new dataset. When a card is built on a DataFusion dataset, the data is then calculated "on the fly" or as needed. Because no physical dataset is created, it can't be an input to Magic ETL or DataFlows. There is potential to make this work in the future (i.e. being able to say, "create an actual new dataset from this datafusion"), but nothing like that currently.
3
Answers
-
Currently it's not possible to have a DataFusion dataset as an input to Magic ETL. The reasoning, without getting to technical, is that a DataFusion is essentially only creating a view or "description" of how datasets should be joined, without actually running any calculations or physically creating a new dataset. When a card is built on a DataFusion dataset, the data is then calculated "on the fly" or as needed. Because no physical dataset is created, it can't be an input to Magic ETL or DataFlows. There is potential to make this work in the future (i.e. being able to say, "create an actual new dataset from this datafusion"), but nothing like that currently.
3 -
yes, thank you
1 -
"There is potential to make this work in the future (i.e. being able to say, "create an actual new dataset from this datafusion"), but nothing like that currently."
Currently sitting in the Data Manipulation class at Domopalooza. Everyone agrees this is a much needed feature.
1 -
0
Categories
- All Categories
- 1.9K Product Ideas
- 1.9K Ideas Exchange
- 1.6K Connect
- 1.3K Connectors
- 305 Workbench
- 6 Cloud Amplifier
- 9 Federated
- 3K Transform
- 106 SQL DataFlows
- 646 Datasets
- 2.2K Magic ETL
- 4K Visualize
- 2.5K Charting
- 774 Beast Mode
- 75 App Studio
- 43 Variables
- 728 Automate
- 186 Apps
- 467 APIs & Domo Developer
- 61 Workflows
- 14 DomoAI
- 40 Predict
- 17 Jupyter Workspaces
- 23 R & Python Tiles
- 403 Distribute
- 117 Domo Everywhere
- 277 Scheduled Reports
- 9 Software Integrations
- 136 Manage
- 133 Governance & Security
- 8 Domo Community Gallery
- 44 Product Releases
- 12 Domo University
- 5.4K Community Forums
- 40 Getting Started
- 30 Community Member Introductions
- 112 Community Announcements
- 4.8K Archive