How to reduce SQL Dataflow runtime?

Prajju
Prajju Member
edited March 2023 in Scheduled Reports

Hi,

 

I was wondering if there are any effecient ways to reduce the MySQL Dataflow runtimes?

Please provide any tips/tricks.

 

Thanks in advance.

Best Answer

  • AS
    AS Coach
    Answer ✓

    If you're using transforms in multiple stages, you can create new transforms, without outputs, and create indexes in them, like:

     

    alter table `transform_data_1` add index (`column_name1`,`column_name2`)

    Aaron
    MajorDomo @ Merit Medical

    **Say "Thanks" by clicking the heart in the post that helped you.
    **Please mark the post that solves your problem by clicking on "Accept as Solution"

Answers

  • AS
    AS Coach

    What have you currently implemented?

     

    The first option is to ensure you're properly indexing your inputs as well as your transforms.

    A second option would be to reduce the number of columns your inputs and transforms have.

     

    Aaron
    MajorDomo @ Merit Medical

    **Say "Thanks" by clicking the heart in the post that helped you.
    **Please mark the post that solves your problem by clicking on "Accept as Solution"
  • Prajju
    Prajju Member

    I am joining 3 different tables and performing calculations on the 3 columns in the 3 different datasets.

    I have indexed columns which are involved in the join.

    How to index on the transforms?

  • AS
    AS Coach
    Answer ✓

    If you're using transforms in multiple stages, you can create new transforms, without outputs, and create indexes in them, like:

     

    alter table `transform_data_1` add index (`column_name1`,`column_name2`)

    Aaron
    MajorDomo @ Merit Medical

    **Say "Thanks" by clicking the heart in the post that helped you.
    **Please mark the post that solves your problem by clicking on "Accept as Solution"
This discussion has been closed.

Categories