How do I filter duplicative entries when two date fields are involved? (entry & modified dates)

RocketMichael
RocketMichael Member
in Magic ETL

Hey Domo Smarties!

[Annotated Screen Cap Below]

What's the best way to filter out duplicate rows according to the latest modified date in an ETL?

Desired Result:

One unique entry by "start date" and "model" with the latest "ob modified date".


Screen Shot 2022-12-21 at 10.14.41 AM.png


Tagged:

Best Answer

  • RocketMichael
    RocketMichael Member
    Answer ✓

    Makes total sense. I started to go into group functions and knew the data wouldn't be accurate... and paused. One more step to re-join with the original data will do the trick!

    Thanks so much!

Answers

  • GrantSmith
    GrantSmith Coach

    You could use a Magic ETL 2.0 dataflow and group your data on the start_date, model and inventory_units and select the MAX ob_modified_date then do an inner join from your original input dataset and the output of the group by based on the start_date, model, inventory_units and ob_modified_date.

    **Was this post helpful? Click Agree or Like below**
    **Did this solve your problem? Accept it as a solution!**
  • RocketMichael
    RocketMichael Member
    Answer ✓

    Makes total sense. I started to go into group functions and knew the data wouldn't be accurate... and paused. One more step to re-join with the original data will do the trick!

    Thanks so much!

Categories