Is there an alternative of distinct on for redshift?

user084060
user084060 Member
in Magic ETL

Hey,

I am trying to build a dataflow using redshift but distinct on is not supported.


It's a simple query:

(SELECT distinct on ("id")

var_list.*

from "indicator" as ind

order by "id", "rank" desc)


Any suggestions for workarounds?

Tagged:

Answers

  • jaeW_at_Onyx
    jaeW_at_Onyx Coach

    @user084060 

    SELECT DISTINCT "id" FROM "table"

    is supported in redshift, but you've got some interesting syntax going on there, so I'm not sure I understand what you're trying to get done.


    Side note, if all you need is a SELECT DISTINCT, Magic 2.0 will probably outperform a Redshift transform. I strongly recommend you take a look. With updates to the engine, and inclusion of the Add Formulas tile, most of the objections SQL developers had against Magic 1.0 ... don't really hold anymore.


    And the performance improvements and increased consistency in dataflow execution times makes it well worth the effort to refactor.

    If you don't have Magic 2.0 in your instance, contact your CSM and ask them to enable the beta.

    Jae Wilson
    Check out my 🎥 Domo Training YouTube Channel 👨‍💻

    **Say "Thanks" by clicking the ❤️ in the post that helped you.
    **Please mark the post that solves your problem by clicking on "Accept as Solution"
  • user084060
    user084060 Member

    @jaeW_at_Onyx I am basically trying to get the highest rank for every id, but the rank can vary and be anything between 1 to 10 (so I cannot solve it through a where statement with rank=..). And I am looking at id and addtional id features.

  • jaeW_at_Onyx
    jaeW_at_Onyx Coach

    Sort your rank DESCENDING and then filter WHERE rank = 1.

    Jae Wilson
    Check out my 🎥 Domo Training YouTube Channel 👨‍💻

    **Say "Thanks" by clicking the ❤️ in the post that helped you.
    **Please mark the post that solves your problem by clicking on "Accept as Solution"

Categories