MagicETL Recipes

DomoDork · July 31

We all have ETLs where we use common logic over and over again. I propose an idea to enhance MagicETL with something called 'Recipes'. A user could create a new recipe within Magic, add a series of tiles into it that do some common logical steps and save it with a given name (e.g.. 'Strip Whitespace and Split String'. Once saved, you would see your recipe on the tile selection pane in magic. When you drag that recipe onto the MagicETL canvas, all tiles within that recipe are recreated and pre-configured (minus column names etc).

Saving a recipe would tie that recipe to the user who created it so that any new MagicETLs that person creates has those recipes readily available in their tile selection pane. Similar to beast modes, you could then ask for that recipe to be certified. Once certified, any user creating MagicETLs will see those certified recipes.

This would allow organizations to build a library of Magic tips and tricks if you will… and help developers more rapidly work through common logical design patterns when developing ETLs.

rolyat · August 7

That would be nice. At least you can currently use the "Copy to Clipboard" function to pull tiles from one ETL to another.

I have a browser extension called Domo Helper that I will be releasing soon, I could probably make this feature using the copy to clipboard function, I would add a button to below the Copy to Clipboard, called Save Recipe or Save Tile set or something, when clicked it would use the copy to clipboard function, save the copied tile set json data into the extensions local storage and prompt you to give it a name. Then you could click any saved tile set from the extension and it would automatically be inserted into the magic ETL flow.

MichelleH · August 7

This is a cool idea! It would be great for standardizing logic and helping new users get up to speed.

rolyat · August 8

https://community-forums.domo.com/main/discussion/comment/97687#Comment_97687

All right, I got carried away and made this a reality.

https://youtu.be/DRm2Z6gEJYY

If you want to be notified when the extension is available for download in a 1-3 weeks just sign up to be notified on my website: https://blueprintcoders.com/domo-helper/

DomoDork · August 8

@haydentaylor - you're a madman! It's amazing to see an idea come to life so quickly :) Amazing work.

rolyat · August 8

https://community-forums.domo.com/main/discussion/comment/97704#Comment_97704

Thanks! I was looking through your other product ideas since this one was so good. I think I can also make this one into a feature as well. I've been bothered how hard it is to trace back columns in a large ETL to this is a great idea, plus I'll try to pair it with some kind of search, trace function to highlight where a column originates from in any point in the ETL plus anywhere it is modified.

https://community-forums.domo.com/main/discussion/60129/colorize-column-headers-in-magicetl-previews-based-on-connecting-line-color

MagicETL Recipes

Submitted and awaiting review from product team · Last Updated July 31

Comments

Categories