Skip to content

    • SQL DataFlows

    SQL DataFlows

    MYSQL Time

    Member
    in SQL DataFlows

    Hi,

    Is there a way to cut down the time of MYSQL dataflows?

    I have a dataflow that takes 6 hours for 5 million rows of data and would love to find a way to cut that down.

    Best Answer

    • Coach
      Answer ✓

      You can try and implement indexes in your MySQL dataflow however I'd HIGHLY recommend rewriting your MySQL dataflow in Magic ETL as MySQL is sequential (it has to wait for each dataset load or step to complete before it can move to the next one) compared to Magic's sequential processing (can do many steps simultaneously).

      I've seen improvements of over 90% when converting MySQL to Magic ETL.

      **Was this post helpful? Click Agree or Like below**
      **Did this solve your problem? Accept it as a solution!**

    Answers

    • Coach
      Answer ✓

      You can try and implement indexes in your MySQL dataflow however I'd HIGHLY recommend rewriting your MySQL dataflow in Magic ETL as MySQL is sequential (it has to wait for each dataset load or step to complete before it can move to the next one) compared to Magic's sequential processing (can do many steps simultaneously).

      I've seen improvements of over 90% when converting MySQL to Magic ETL.

      **Was this post helpful? Click Agree or Like below**
      **Did this solve your problem? Accept it as a solution!**
    • Member

      I will give that a try, thank you.

    Welcome!

    It looks like you're new here. Members get access to exclusive content, events, rewards, and more. Sign in or register to get started.
    Sign In

    Welcome!

    It looks like you're new here. Members get access to exclusive content, events, rewards, and more. Sign in or register to get started.
    Sign In

    Quick Links

    Categories