Magic ETL

Magic ETL

Multiple singe day events grouped as single consecutive day events

I am trying to figure out a way, in Magic ETL, to take multiple single day events and group them when there is break in consecutive days.

Example, if the results show the 1st, 2nd, 3rd, 5th, 6th, 9th, 10th and 11th, I need three results, not 8. I want to see begin and end dates of 1st-3rd, 5th-6th, and 9th-11th.

Any thoughts?

Welcome!

It looks like you're new here. Members get access to exclusive content, events, rewards, and more. Sign in or register to get started.
Sign In

Best Answer

Answers

  • edited January 2024

    Classic gap and island analysis. Here is an article introducing the topic and how to accomplish what you are trying to do: Site faviconIntroduction to Gaps and Islands Analysis - Simple Talk

    An alternative way in MagicETL would be to use a lag function and have some logic that if the previous value is not the current value minus one then that current value must be a new start to an island (the 5th in your example). Similarly you could do a lead function and anytime the future value isn't the current value plus one then it is the start of an island (the 6th in your example).

    I think the linked article is the cleanest way to do this in a MySQL dataflow.

    If I solved your problem, please select "yes" above

  • Member
    edited January 2024

    Thank you @ColemenWilson, but could you expand on the MagicETL method? I've been trying to use the Rank & Window with the previous date (lag), but not able to get to the grouping I'm hoping for. I need to somehow carry a constant down the rows until it sees that the datediff is >1, then a new constant until >1, etc.

  • Answer ✓

    Walked through how to do it here: Site faviconLoom | Free Screen & Video Recording Software



    This is modified from what I previously suggested and doesn't require lag and lead functions. Big thanks to @AustinHarrison for the idea of a more simple way to do this using row numbers!

    If I solved your problem, please select "yes" above

Welcome!

It looks like you're new here. Members get access to exclusive content, events, rewards, and more. Sign in or register to get started.
Sign In