Joining based on conditions in Magic ETL

user022825

Can we make joins based on conditions in Magic ETL? e.g. I want to replicate the below in a join in magic ETL. Can I do it?

 

SELECT

dataset_1.*

dataset_2.*

FROM dataset_1

LEFT JOIN `dataset_2` on
(dataset_2.id` IS NULL
AND `dataset_1`.`Day` >= COALESCE(dataset_2.date`,dataset_1.date`) )

GrantSmith

Hi @user022825 

Currently you can't exactly do conditional joins with Magic ETL. You'd need to create a single (or multiple) columns you can join on and then do your filtering / conditional join logic after the join has been done.

 

Simply put there isn't an easy way of doing this type of join with Magic ETL. There are possible other ways but it would involve greatly increasing your dataset size thus increasing your processing speed.

 

jaeW_at_Onyx

if the datasets are of reasonable size you can do a cross apply on dataset A to dataset B (join where 1 = 1) then use a FILTER tile to remove rows that don't meet your JOIN criteria. 

 

It's not great, but it gets the job done.