How to update field if value is different than existing?

BMilner · January 17

Hello,

I feel like this should be straightforward, but I can't figure it out. During our master ETL, I have a lookup sheet that adds fields to the dataset (e.g. Account Owner, Region, Account tier (A, B, or C). However, if an account owner changes and I make a change to the lookup sheet, it's only updating new records since we are updating the dataset by batch. All prior records retain the old value. Is there an easy way to update all records without rerunning the entire dataset?

I'd even be fine with filtering the output dataset and remapping, but it looks like that applies the change to all records and not just the filtered rows.

Sorry, very new to this.

Thanks, Brian

ST_-Superman-_ · January 17

@BMilner

I would recommend joining this data in using a dataset view. These joins happen much faster than ETLs (near instant). The downside here would be that you would need to change any cards or content you have built to a new dataset.

ColemenWilson · January 17

Pretty sure you'll have to re-run the entire dataset. Is there a reason you are against doing that one time to cleanup the account owners?

BMilner · January 17

Thanks for the response. Last time I tried to do that, it timed-out after 24 hours.

ColemenWilson · January 17

Gotcha. Have you looked at using Magic ETL and/or Datafusion to speed things up and prevent a timeout?

ST_-Superman-_ · January 17

@BMilner

I would recommend joining this data in using a dataset view. These joins happen much faster than ETLs (near instant). The downside here would be that you would need to change any cards or content you have built to a new dataset.

How to update field if value is different than existing?

Best Answer

Answers

Categories