Best practice for removing historical values

renee12345
renee12345 Member
in Datasets

I have a snapshot dataflow that has captured bad values. I only need to filter out those values that don't match the current format. What is the best practice for this?

Best Answers

  • DavidChurchman
    DavidChurchman Coach
    Answer ✓

    You can add multiple conditions to a filter tile, then there is the option of "all" or "any" of the rules to be true. Basically, logical "AND" or logical "OR". If it's more complex than that, you can create a "formula rule" within the filter tile. KB article about filter formula

    image.png

    Please 💡/💖/👍/😊 this post if you read it and found it helpful.

    Please accept the answer if it solved your problem.

  • ColemenWilson
    ColemenWilson Coach
    Answer ✓

    Yep, you would do what @DavidChurchman said and set each of those 3 conditions as conditions in the filter with "Include rows that meet ALL of the following rules" set.

    If I solved your problem, please select "yes" above

Answers

  • ColemenWilson
    ColemenWilson Coach

    Can you search for the rows that don't match the current format in a dynamic way? If so you could just apply a filter tile to the ETL to remove the rows that don't match. How do you identify the bad rows?

    If I solved your problem, please select "yes" above

  • renee12345
    renee12345 Member

    @colemenwilson do you know the best way to utilize the filter formula tile to exclude multiple values? ex:

    column1 != 'one'

    column1 != 'two'

    column1 != 'three'

    Is there a way to do this in one formula?

  • DavidChurchman
    DavidChurchman Coach
    Answer ✓

    You can add multiple conditions to a filter tile, then there is the option of "all" or "any" of the rules to be true. Basically, logical "AND" or logical "OR". If it's more complex than that, you can create a "formula rule" within the filter tile. KB article about filter formula

    image.png

    Please 💡/💖/👍/😊 this post if you read it and found it helpful.

    Please accept the answer if it solved your problem.

  • ColemenWilson
    ColemenWilson Coach
    Answer ✓

    Yep, you would do what @DavidChurchman said and set each of those 3 conditions as conditions in the filter with "Include rows that meet ALL of the following rules" set.

    If I solved your problem, please select "yes" above

Categories