Redshift Flow
We have a redshift flow that runs daily. The flow has past three years of data from three different flows filtered by the date. Recently some changes were made to the historical data and I cannot see those reflected in the redshift flow. However, if I write the same code in a separate Redshift dataflow it works. Can anyone please help?
Best Answer
-
If it's not recursive when last did dataflow run? That's the only reason I can think of aside from a transform within the dataflow excluding 2022 data and thus preventing the changes that have been made to 2022 data from appearing in the dataflow output.
0
Answers
-
Do you see the changes reflected in the 3 different flows powering your redshift dataflow? Were these changes to existing columns or did these changes result in new columns needing to be added? Does the date filter encompass the full date range affected by the recent changes? It sounds like your dataflow is recursive. Changes to historical data in recursive dataflows aren't always straightforward. You may consider rebuilding this in ETL and partitioning the data, possibly of the same date field.
0 -
Hey there! Yes, I see changes reflected in the three flows. The changes were to one of the flows. The raw data was replaced entirely for 2022. It isn't a recursive flow.
0 -
If it's not recursive when last did dataflow run? That's the only reason I can think of aside from a transform within the dataflow excluding 2022 data and thus preventing the changes that have been made to 2022 data from appearing in the dataflow output.
0
Categories
- All Categories
- 1.8K Product Ideas
- 1.8K Ideas Exchange
- 1.6K Connect
- 1.2K Connectors
- 300 Workbench
- 6 Cloud Amplifier
- 9 Federated
- 2.9K Transform
- 101 SQL DataFlows
- 623 Datasets
- 2.2K Magic ETL
- 3.9K Visualize
- 2.5K Charting
- 748 Beast Mode
- 60 App Studio
- 41 Variables
- 690 Automate
- 177 Apps
- 455 APIs & Domo Developer
- 48 Workflows
- 10 DomoAI
- 36 Predict
- 15 Jupyter Workspaces
- 21 R & Python Tiles
- 397 Distribute
- 114 Domo Everywhere
- 276 Scheduled Reports
- 7 Software Integrations
- 127 Manage
- 124 Governance & Security
- 8 Domo Community Gallery
- 38 Product Releases
- 10 Domo University
- 5.4K Community Forums
- 40 Getting Started
- 30 Community Member Introductions
- 109 Community Announcements
- 4.8K Archive