Preview flow "from here" option?

DHo

I think I know the answer, but I thought I'd check anyway …

Is there any hack to cause a Magic ETL preview to run from a certain point in the flow forward?

I'm working on a flow that starts with some heavy sources and groups down before performing some merges and other heavy actions. I would LOVE to preview from a point after the first round of groupings for quicker testing of my fumbling attempts to get what I'm after.

😬

MarkSnodgrass

As @GrantSmith said, this feature doesn't currently exist. However, you can do what you want to do by breaking up your ETL into smaller ETLs for testing. Then, once you have it all sorted it out, you can copy and paste the tiles back into your main one so that you have a single ETL for production.

I would create a ETL Phase 1, for example, that does all the grouping work that you are referring to and execute that ETL. Then, create ETL Phase 2 and attach your Phase 1 output dataset as the input dataset of Phase 2. This should allow you test a bit quicker.

GrantSmith

Currently no, this isn't an option but It'd be a great recommendation for the Ideas Exchange.

MarkSnodgrass

As @GrantSmith said, this feature doesn't currently exist. However, you can do what you want to do by breaking up your ETL into smaller ETLs for testing. Then, once you have it all sorted it out, you can copy and paste the tiles back into your main one so that you have a single ETL for production.

I would create a ETL Phase 1, for example, that does all the grouping work that you are referring to and execute that ETL. Then, create ETL Phase 2 and attach your Phase 1 output dataset as the input dataset of Phase 2. This should allow you test a bit quicker.

DHo

Great suggestion, @MarkSnodgrass 👌 I'll use that as a workaround.