Does RedShift offer better performance?

cwolman
cwolman Contributor
in Magic ETL

I am currently using MySQL data flows and indexing the join fields appropriately but performance seems underwhelming when dealing with millions of rows and/or GB sized data sets.  Would I see a performance benefit utilizing RedShift?  I began looking into the RedShift documentation and the first difference of note was indexes vs sort key.  How do you handle this difference in Domo? 

 

Thank you in advance for your responses.


-----------------
Chris

Best Answer

  • AS
    AS Coach
    Answer ✓

    We also use Redshift exclusively.  Our datasets range from dozens of rows to dozens of millions of rows.

     

    Some benefits for my perspective: 

    1) No indexing, less maintenance

    2) Faster processing on larger datasets

    3) Common table expressions, window functions, etc (mysql didn't have this previously, though I've heard that may have changed)

     

    I used to be all about MySQL when I worked for Domo, but that indexing step really bugged me.  I don't want that hassle so for me it's a big win to be free of query optimization like that.

    Aaron
    MajorDomo @ Merit Medical

    **Say "Thanks" by clicking the heart in the post that helped you.
    **Please mark the post that solves your problem by clicking on "Accept as Solution"

Answers

  • Chronos
    Chronos Member

    Is anyone able to help out with this request?

  • RGranada
    RGranada Contributor

    Hi,

     

    From my experience and if you are handling justifiable amounts of data redshift will offer you a considerable boost in performance.

     

    Actually, all of my 200+ dataflows are exclusively redshift.

     

    I would say that for anything over 250.000 rows go with Redshift (actually I am using it even for fewer quantities of data).

     

    Hope this helps.

    Ricardo Granada 

    MajorDomo@Lusiaves

    **If the post solves your problem, mark it by clicking on "Accept as Solution"
    **You can say "Thank you" by clicking the thumbs up in the post that helped you.
  • AS
    AS Coach
    Answer ✓

    We also use Redshift exclusively.  Our datasets range from dozens of rows to dozens of millions of rows.

     

    Some benefits for my perspective: 

    1) No indexing, less maintenance

    2) Faster processing on larger datasets

    3) Common table expressions, window functions, etc (mysql didn't have this previously, though I've heard that may have changed)

     

    I used to be all about MySQL when I worked for Domo, but that indexing step really bugged me.  I don't want that hassle so for me it's a big win to be free of query optimization like that.

    Aaron
    MajorDomo @ Merit Medical

    **Say "Thanks" by clicking the heart in the post that helped you.
    **Please mark the post that solves your problem by clicking on "Accept as Solution"

Categories