How to update field if value is different than existing?

BMilner

Hello,

I feel like this should be straightforward, but I can't figure it out. During our master ETL, I have a lookup sheet that adds fields to the dataset (e.g. Account Owner, Region, Account tier (A, B, or C). However, if an account owner changes and I make a change to the lookup sheet, it's only updating new records since we are updating the dataset by batch. All prior records retain the old value. Is there an easy way to update all records without rerunning the entire dataset?

I'd even be fine with filtering the output dataset and remapping, but it looks like that applies the change to all records and not just the filtered rows.

Sorry, very new to this.

Thanks, Brian

ST_Superman

@BMilner

I would recommend joining this data in using a dataset view. These joins happen much faster than ETLs (near instant). The downside here would be that you would need to change any cards or content you have built to a new dataset.

ColemenWilson

Pretty sure you'll have to re-run the entire dataset. Is there a reason you are against doing that one time to cleanup the account owners?

BMilner

Thanks for the response. Last time I tried to do that, it timed-out after 24 hours.

ColemenWilson

Gotcha. Have you looked at using Magic ETL and/or Datafusion to speed things up and prevent a timeout?

ST_Superman

@BMilner

I would recommend joining this data in using a dataset view. These joins happen much faster than ETLs (near instant). The downside here would be that you would need to change any cards or content you have built to a new dataset.