-
Magic ETL Organization Tiles
Would be nice to have a feature to add a box around several tiles in order to organise better the dataflow. Sometimes we have a complex dataflow with several business logic and a lot of tiles, and it would be easier to identify the several steps with that boxes. An example of that is like the one in Unreal Engine:
-
Pass Variables into a MagicETL from Workflows.
With the new workflow features on the horizon, I think it would be immeasurably valuable to pass output variables from workflows into Magic ETLs since workflows can execute ETLs already. For example: An 'audit' workflow is triggered that evaluates a dataset for various criteria. As part of the evaluation it sets output…
-
Add Hints to Deduplicate Tile
When I ingest the customer table from our POS, for example, and compare individual customer records to the last ingestion, I want to keep the older of the two records if selected fields show the records to be duplicates otherwise. The Deduplicate tile is the obvious choice to use, but appears to choose at random which…
-
Option to add prefix in Magic ETL Joins
The Pandas library and Power Query have this option, it's the option to add the table name as a prefix when joining tables. Huge benefits when working with system-generated datasets for example that can have 40+ columns with similar names in each table. The workarounds mentioned here don't quite cut it - if you're used to…
-
Sequence containers for magic etl or data flows
anyway of creating sequence containers for magic etl or dataflows we have many many dataflows and you can easily get lost. Since you cannot group them in a project folder. A sequence container like in ssis were we can group all the etls would be ideal.
-
Actual Run time listed in Lineage Tab
The last run time shows as “hours ago”. This needs to be the actual time the process ran so tracking down issues is much easier.
-
Allow Window Functions in MagicETL Formula Tile
It would be helpful to be able to write MagicETL formulas using window functions, similar to in beast mode. While the Rank & Window tile is often sufficient, there are some use cases where having this feature in the Formula tile would simplify the dataflow steps: Using a sum window function with unbounded rows This does…
-
Allowing comments on ETL schedule changes
We frequently have to disable automation on our Magic ETLs when troubleshooting issues. Currently there is no commenting option as exists when you save edits to the ETL logic. We need to track for auditing purposes why we disabled and then re-enabled automation and need to tie it to an incident number. If we had the…
-
Magic ETL - Undo Button
Can we add an Undo Button to Magic ETL. This would be especially helpful when we accidentally delete a tile when clicking to drag it.
-
Links to Input/Output Datasets in Magic ETL Editor
Oftentimes I find myself having to open a new tab to navigate to the various input and output datasets when in the editing canvas for a Magic ETL. It would be nice if there were a link that would open the dataset in a new tab somewhere on the configuration dialog that appears after selecting the tile.
-
Settings for Magic ETL - only owner & admins can edit this Dataflow
I just saw that the redshift and mySQL dataflows have an option under settings for "Only the owner and administrators can edit this DataFlow". This should also be available in Magic ETL 2.0