Aggregate and ETL question

A2024 · 2023-06-11T01:19:40+00:00

There was an error rendering this rich post.

A2024

Hello all, I have the below workflow in my Magic ETL. Just posting a sample version below.

My question is - I have a ProductID that I join from both the data sources and then do a group by and output to a file.

But I also need to know Total count of Product ID or row count from TEST spreadsheet source. Is there a way I capture the Total count and get that to the same output ?

I can do this in a separate card and display in dashboard. But I want to get that in the same workflow that I am using for my main dashboard.

Please help

Find more posts tagged with

SQL

Magic ETL

Accepted answers

GrantSmith

If you can have all of the records from your top dataset in your output dataset then you can do a left join on your dataset instead of an inner join. Then in your card do a count of the id file from your TEST dataset to get the total records that match

if you just want the entire count then I’d recommend running your TEST dataset through a group by tile and count in there and output it to a separate dataset

All comments

GrantSmith

if you just want the entire count then I’d recommend running your TEST dataset through a group by tile and count in there and output it to a separate dataset

A2024

Thanks @GrantSmith for your quick response. I will need to do a matching on both the sources to get just the matched records. So I will need an inner join.

Yes Thanks @GrantSmith . I will add a Group tile and output to a different dataset.

The only issue I have with that approach is, in the dashboard I want to see both the above outputs together in a tabular form or pivot chart. Do you think this will be doable?