Why are my filters not working in Magic ETL?

damen · 2022-10-05T16:32:02+00:00

There was an error rendering this rich post.

Hi all,

I'm confused as to why my magic ELT filters are not working properly. Is it something I am doing wrong? How do I go about fixing this?

In the first picture, I am in the data set and have set a filter on the 'time_stamp' column and its filtering the data just fine.

For whatever reason, when I go to magic ETL and do the same thing with a tile, it wont spit out any data. Any recommendations?

Find more posts tagged with

Magic ETL

Accepted answers

All comments

MichelleH

Hi @damen does it return no rows when you save and run the dataflow, or just when you run a preview? If you haven't done so already, try running the entire dataflow and see if you get different results.

Previews do not process every row in order to save processing power, so it is possible that the more recent rows are not being included in your preview. You can also adjust how many rows your want to include in your preview using the drop-down on the top of the ETL editor.

damen

@MichelleH That worked. But it was doing both. I tried to save and run it and it returned 0 rows with that same filter. I just changed the row limit and although there aren't even 10k rows (thats why i was trying to filter, to reduce the rows) changing the view to 400k worked. Weird. Thank you tho!!

MichelleH

@damen That's definitely odd! I'm glad it worked!

damen

@MichelleH Ok new question

So I am trying to recreate this pivot table but do it in a way that it just creates the data table using an ETL

Here is what i have done so far

Here is what I am getting

How come the formula that is supposed to be grouped by instrument_name is returning a 100%?

I have tried to group by fallout_reason as well and that doesnt change the withdraw_percent numbers

RobSomers

@damen In your formula, instead of doing COUNT('withdrawn'), do this:

SUM(case when 'fallout_reason' = 'withdrawn' then 1 else 0 end)

As you have it now, the numerator is essentially creating a "column" with the string 'withdrawn' in every row, so the numerator is doing the same thing as your denominator. You need to identify the column your aggregation should be looking at and what the condition is.

jaeW_at_Onyx

@damen to be more specific, you should (probably) impelment the metric @RobSomers is describing as a Beast Mode in Analyzer.

Typically we recommend against preaggregating datasets in ETL if we don't have to.

damen

@RobSomers Awesome. I don't quite understand that 2nd part but I used that syntax and it worked well. Thank you both