Block tiles in Magic ETL to simplify complex dataflows

KevinNorman
KevinNorman Member
edited August 2023 in Magic ETL Ideas

"Block Tiles" are larger tiles that contain a string of tiles inside. Block tiles could make large Magic ETLs (100+ tiles) easier to document, and much easier to read and understand.

These tiles would contain a group of tiles that have a single output tile (not an output dataset) to make for an easy way to connect block tiles to regular ones.

Benefits:

  • Much easier time for new employees who need to understand what these ETLs are doing.
  • Documentation of these ETLs could be mostly self-contained instead of needing to document elsewhere.
  • Help existing engineers compartmentalize the origin of logic within an ETL, making for an easier and faster method of troubleshooting.

Here's a 4-minute video to help understand the impact this feature would have.

10
10 votes

Active · Last Updated