Redshift Flow
We have a redshift flow that runs daily. The flow has past three years of data from three different flows filtered by the date. Recently some changes were made to the historical data and I cannot see those reflected in the redshift flow. However, if I write the same code in a separate Redshift dataflow it works. Can anyone please help?
Best Answer
-
If it's not recursive when last did dataflow run? That's the only reason I can think of aside from a transform within the dataflow excluding 2022 data and thus preventing the changes that have been made to 2022 data from appearing in the dataflow output.
0
Answers
-
Do you see the changes reflected in the 3 different flows powering your redshift dataflow? Were these changes to existing columns or did these changes result in new columns needing to be added? Does the date filter encompass the full date range affected by the recent changes? It sounds like your dataflow is recursive. Changes to historical data in recursive dataflows aren't always straightforward. You may consider rebuilding this in ETL and partitioning the data, possibly of the same date field.
0 -
Hey there! Yes, I see changes reflected in the three flows. The changes were to one of the flows. The raw data was replaced entirely for 2022. It isn't a recursive flow.
0 -
If it's not recursive when last did dataflow run? That's the only reason I can think of aside from a transform within the dataflow excluding 2022 data and thus preventing the changes that have been made to 2022 data from appearing in the dataflow output.
0
Categories
- All Categories
- 1.7K Product Ideas
- 1.7K Ideas Exchange
- 1.5K Connect
- 1.2K Connectors
- 295 Workbench
- 6 Cloud Amplifier
- 8 Federated
- 2.8K Transform
- 97 SQL DataFlows
- 608 Datasets
- 2.1K Magic ETL
- 3.8K Visualize
- 2.4K Charting
- 710 Beast Mode
- 50 App Studio
- 39 Variables
- 668 Automate
- 170 Apps
- 446 APIs & Domo Developer
- 45 Workflows
- 7 DomoAI
- 33 Predict
- 13 Jupyter Workspaces
- 20 R & Python Tiles
- 391 Distribute
- 111 Domo Everywhere
- 274 Scheduled Reports
- 6 Software Integrations
- 116 Manage
- 113 Governance & Security
- Domo Community Gallery
- 31 Product Releases
- 9 Domo University
- 5.3K Community Forums
- 40 Getting Started
- 30 Community Member Introductions
- 103 Community Announcements
- 4.8K Archive