Allow Virtual DataSets as Inputs to Redshift / MySQL Dataflows

We use the Virtual DataSets toolkit item to populate our secondary Domo instance with data from our primary Domo instance. Domo calls the resulting virtual dataset a federated dataset (the data provider is "domo-federateddata"). Unfortunately we cannot use the federated datasets as inputs to Redshift / MySQL ETL dataflows. Domo does not allow a federated dataset to be an ETL input unless it is Snowflake federated dataset.
This limitation makes it difficult to test ETL changes, which is the goal of our secondary / dev instance.
The recommended alternative is to use the DataSet Copy connector, but DataSet Copy can only update the destination instance on a schedule, whereas federated / virtualized datasets update the data in the destination instance immediately whenever the data in the source instance changes. Relying on DataSet Copy to mirror data between instances creates opportunity for a dataset to differ between instances, meaning an ETL that is identical between instances will yield different outputs due to differences in the input dataset. Depending on the frequency at which the ETL runs, the two instances may never produce the same output. Again, this limitation makes it difficult to test ETL changes, which is the entire goal of our secondary / dev instance.
Categories
- 10.5K All Categories
- 7 Connect
- 917 Connectors
- 250 Workbench
- 466 Transform
- 1.7K Magic ETL
- 69 SQL DataFlows
- 477 Datasets
- 193 Visualize
- 252 Beast Mode
- 2.1K Charting
- 11 Variables
- 17 Automate
- 354 APIs & Domo Developer
- 89 Apps
- 3 Workflows
- 20 Predict
- 5 Jupyter Workspaces
- 15 R & Python Tiles
- 246 Distribute
- 62 Domo Everywhere
- 243 Scheduled Reports
- 21 Manage
- 42 Governance & Security
- 173 Product Ideas
- 1.2K Ideas Exchange
- 12 Community Forums
- 27 Getting Started
- 14 Community Member Introductions
- 55 Community News
- 4.5K Archive