How to Remove Duplicate rows in analyzer?

SaloniShah
SaloniShah Member
edited October 2022 in Charting

I have used the LinkedIn connector to export the stats. I see duplicated rows. How do I avoid them?

image.png


Tagged:

Best Answer

  • RobSomers
    RobSomers Coach
    Answer ✓

    @SaloniShah The less than ideal solution is to create an ETL that and use the Remove Duplicates tile, but that doesn't solve the underlying issue. One thing you might try is delete and recreate the connector. If that doesn't solve the issue, it's most likely a bug with the connector and you should put in a support ticket.

    **Was this post helpful? Click Agree or Like below**

    **Did this solve your problem? Accept it as a solution!**

Answers

  • jaeW_at_Onyx
    jaeW_at_Onyx Coach

    @SaloniShah is the data actually duplicated in your dataset? If so that's probably something to be concerned about if it's not intentional.

    Take a look at the connector that's bringing in the data, is it set to APPEND or REPLACE?

    Is the data actually duplicated or are just a few rows duplicates? If you run the connector multiple times do you accumulate a copy of the dataset each time? (if so that implies that you're running an APPEND instead of REPLACE in your connector settings.


    Sometimes Domo's datasets are 'broken', to deduplicate a row, you can use "remove duplicates" tile in Magic ETL or use a rank_window tile and apply a row_number to each row partitioned on date then apply a filter. (remove duplicates is probably conceptually easier to understand, rank_window is more precise).

    Jae Wilson
    Check out my 🎥 Domo Training YouTube Channel 👨‍💻

    **Say "Thanks" by clicking the ❤️ in the post that helped you.
    **Please mark the post that solves your problem by clicking on "Accept as Solution"
  • SaloniShah
    SaloniShah Member

    Hello @jaeW_at_Onyx , thanks for responding.

    My data is set to Replace. Duplicates is definitely not the issue.

    I'm attaching screenshots where in the Data tab I see single rows but when we go to visualize tab we see it duplicated. Example- 305 likes in May 2022.


    DOMOL.PNG
    DOMOLL.PNG


  • MichelleH
    MichelleH Coach

    @SaloniShah Based on your screenshots, those duplicates don't appear to actually exist in your dataset. Try removing the sorting in your card. Sometimes unnecessary sort fields can cause cards to not display correctly.

  • SaloniShah
    SaloniShah Member
    image.png

    Hey @MichelleH , thanks for your response. I did remove the sorting. Still, I see these duplicates. What else can the issue be?

  • RobSomers
    RobSomers Coach

    @SaloniShah Looking back at what you posted about not seeing duplicates in your dataset in the data tab, I actually do see duplicate rows, they just aren't right next to each other:

    image.png


    **Was this post helpful? Click Agree or Like below**

    **Did this solve your problem? Accept it as a solution!**

  • SaloniShah
    SaloniShah Member

    @RobSomers I just noticed. But I did check my dataset its getting replaced & not Appended. Why would it create duplicates ?

  • RobSomers
    RobSomers Coach
    Answer ✓

    @SaloniShah The less than ideal solution is to create an ETL that and use the Remove Duplicates tile, but that doesn't solve the underlying issue. One thing you might try is delete and recreate the connector. If that doesn't solve the issue, it's most likely a bug with the connector and you should put in a support ticket.

    **Was this post helpful? Click Agree or Like below**

    **Did this solve your problem? Accept it as a solution!**

  • SaloniShah
    SaloniShah Member

    I recreated the connector and it still shows duplicates. I would try creating a ticket for support.

    Thank you for your help.

Categories