How to reduce Magic ETL run time

When I'm developing ETLs I like to be a teapot - Short and Stout.

Short

Reduce the dataset size to the smallest you need before doing any more processing. Have a filter tile be your first tile or use a Dataset View to pre-filter your dataset and then use that as your input (this will cause less data to be transferred to Magic ETL engine reducing the execution time. Also by using an Alter Columns or Select Columns to only use the columns you need for the ETL will also help reduce the amount of data the ETL needs to process

Stout

This doesn't mean make your dataset wide with a bunch of columns but we want to be wide and leverage Magic ETL's parallel processing powers. Rather than having a single pipe that you're joining multiple datasets to one at a time do multiple joins or multiple different operations on the datasets before you do a join. For example if I have four tables, customers, orders, order line items, products. Instead of Joining customers to orders and then joining that output to line items and then joining that output to products you'd instead join Customers and orders together in one pipe and at the same time join the line items to the products and then joining those two outputs together. This will allow Domo to do multiple operations at the same time reducing your processing time.