best way to replace a subset of data
I have a large historical dataset to which I append records at the end of each day. I have been asked to correct the distribution of values for a small subset of the data over the past nine months. I can use the data repair tool feature to remove all of the data imports for that time, but this would force me to destroy and rebuild much more data than is necessary. Is there a scalpel I can use to excise and replace the exact records I need to adjust? I do not wish to take a sledge hammer to large portions of the dataset.
Could I perhaps temporarily alter the import SQL and change its type to use a REPLACE strategy and only update for the intended subset of the data, then change the import back into an APPEND import strategy? Or would this drop all of the non-targeted records from the historical table?
If I need to wholesale replace all the imports over the past nine months, then it might be better for me to use a workflow to move all of the earlier data to a separate table, then erase the original table and import the last nine months of data into another table. I could then use another transformation to combine the two tables into a replacement for the original table. This might be a better long-term strategy for maintaining a historical data set. It would allow me to maintain historical data in separate tables that I could then combine for my long-term reports.
Best Answers
-
I would suggest you Magic ETL to clean this up. You can have two filter tiles (one that has the data you need to clean, and the other data that is does not need clean). You can then do your cleaning using a formula tile and then append the two back together and output to a single dataset. You would then build your cards off of this dataset. A basic example would look like this:
**Check out my Domo Tips & Tricks Videos
**Make sure to any users posts that helped you.
**Please mark as accepted the ones who solved your issue.1 -
use @MarkSnodgrass recommendation plus partitioning in Magic
Jae Wilson
Check out my 🎥 Domo Training YouTube Channel 👨💻
**Say "Thanks" by clicking the ❤️ in the post that helped you.
**Please mark the post that solves your problem by clicking on "Accept as Solution"0
Answers
-
I would suggest you Magic ETL to clean this up. You can have two filter tiles (one that has the data you need to clean, and the other data that is does not need clean). You can then do your cleaning using a formula tile and then append the two back together and output to a single dataset. You would then build your cards off of this dataset. A basic example would look like this:
**Check out my Domo Tips & Tricks Videos
**Make sure to any users posts that helped you.
**Please mark as accepted the ones who solved your issue.1 -
use @MarkSnodgrass recommendation plus partitioning in Magic
Jae Wilson
Check out my 🎥 Domo Training YouTube Channel 👨💻
**Say "Thanks" by clicking the ❤️ in the post that helped you.
**Please mark the post that solves your problem by clicking on "Accept as Solution"0
Categories
- All Categories
- 1.8K Product Ideas
- 1.8K Ideas Exchange
- 1.5K Connect
- 1.2K Connectors
- 297 Workbench
- 6 Cloud Amplifier
- 8 Federated
- 2.9K Transform
- 100 SQL DataFlows
- 614 Datasets
- 2.2K Magic ETL
- 3.8K Visualize
- 2.5K Charting
- 729 Beast Mode
- 53 App Studio
- 40 Variables
- 677 Automate
- 173 Apps
- 451 APIs & Domo Developer
- 45 Workflows
- 8 DomoAI
- 34 Predict
- 14 Jupyter Workspaces
- 20 R & Python Tiles
- 394 Distribute
- 113 Domo Everywhere
- 275 Scheduled Reports
- 6 Software Integrations
- 121 Manage
- 118 Governance & Security
- Domo Community Gallery
- 32 Product Releases
- 10 Domo University
- 5.4K Community Forums
- 40 Getting Started
- 30 Community Member Introductions
- 108 Community Announcements
- 4.8K Archive