How to reduce SQL Dataflow runtime?

Prajju

Hi,

 

I was wondering if there are any effecient ways to reduce the MySQL Dataflow runtimes?

Please provide any tips/tricks.

 

Thanks in advance.

AS

If you're using transforms in multiple stages, you can create new transforms, without outputs, and create indexes in them, like:

 

alter table `transform_data_1` add index (`column_name1`,`column_name2`)

AS

What have you currently implemented?

 

The first option is to ensure you're properly indexing your inputs as well as your transforms.

A second option would be to reduce the number of columns your inputs and transforms have.

 

Prajju

I am joining 3 different tables and performing calculations on the 3 columns in the 3 different datasets.

I have indexed columns which are involved in the join.

How to index on the transforms?

AS

If you're using transforms in multiple stages, you can create new transforms, without outputs, and create indexes in them, like:

 

alter table `transform_data_1` add index (`column_name1`,`column_name2`)