I need to walk a date column partitioned by the value of another column (UID) to find the earliest date for that group.
I do not want to collapse the rows.
I've tried a number of approaches to this with a SQL perspective, but I'm starting to think Magic ETL wants me to use a Group by.
Here are a couple of examples of my experimentation to illustrate what I'm trying to do.
min( active_date ) over( partition by acct_id )first_non_null_value( min( active_date ) over( partition by acct_id order by active_date ASC ) )I could group then join to get the value, I was just trying to optimize the flow.
You're correct, Minimum isn't an option, apologies. I did log a new idea to the idea exchange to add this in as it seems to be a glaring omission from that tile:
Really there isn't a way in Magic ETL except for doing a Group By and then joining back to your original dataset based on the grouping keys you used to get the minimum date.
Have you looked into using a Rank & Window til in your Magic ETL to get the minimum date across your column partition?
Unless I've missed something in the Magic ETL R&W implementation, that's just going to populate my new column with a rank number. I want the "first date" value to be populated as the new column's value so I can use that for a later grouping/agg.
You're correct, Minimum isn't an option, apologies. I did log a new idea to the idea exchange to add this in as it seems to be a glaring omission from that tile:
Really there isn't a way in Magic ETL except for doing a Group By and then joining back to your original dataset based on the grouping keys you used to get the minimum date.
Thanks @GrantSmith. That's the work around I'm using. It works, but I wish I could do it all with a single function tile and string together a bunch of SQL statements.
I've been trying to get the hang of Magic ETL, but I may have to switch back to a basic SQL flow for this particular data set.
