Deduplicating a table based on the content of certain columns

Options
Stan_Smith
Stan_Smith Member
in Magic ETL

I have a dataset that has about 20 columns in it. The first column contains ID numbers and a lot of the ID numbers are duplicated multiple times. All the other data in the columns are also duplicated multiple times except one column named last_updated. The last_updated column lists a date.

I would like to de-dupe this dataset and keep the rows with the most recent dates in the last_updated columns. Is there a way to do this?

Tagged:

Comments

  • MarkSnodgrass
    MarkSnodgrass Coach
    Options

    The easiest way to do this is to use the Group By tile in Magic ETL. Add all your columns in the select except for the last_updated column. Add that one to the aggregated column list and choose Max. This will give you the most recent date for each.

    **Check out my Domo Tips & Tricks Videos

    **Make sure to <3 any users posts that helped you.
    **Please mark as accepted the ones who solved your issue.

Categories