-
Magic ETL 2.0 - Remove Duplicates needs an "ADD ALL COLUMNS" option
The title says it all!
-
Allow Window Functions in MagicETL Formula Tile
It would be helpful to be able to write MagicETL formulas using window functions, similar to in beast mode. While the Rank & Window tile is often sufficient, there are some use cases where having this feature in the Formula tile would simplify the dataflow steps: Using a sum window function with unbounded rows This does…
-
Block tiles in Magic ETL to simplify complex dataflows
"Block Tiles" are larger tiles that contain a string of tiles inside. Block tiles could make large Magic ETLs (100+ tiles) easier to document, and much easier to read and understand. These tiles would contain a group of tiles that have a single output tile (not an output dataset) to make for an easy way to connect block…
-
Magic ETL - Allow group by without a grouping column
There are times when I want to calculate aggregated metrics across the entire dataset. Currently this isn't possible without adding a constant value column to group on which I then have to remove later in my pipeline. It'd be much smoother if the Group By tile allowed us to group across the entire dataset without having to…
-
Append Rows Should Default to "Include all Columns"
Can we have the Appends Rows tile default to "Include all Columns" instead of just selecting the columns from the first input?
-
Assign colors to tiles
I would like to be assign colors to tiles: this would make troubleshooting and code reviews much easier. A stretch goal would be to add setting the color of tiles to the options when you select more than one tile in addition to Delete, Duplicate, and Copy To Clipboard. Using this to visually indicate functional sections of…
-
Add Formula Rule to Join Tile in ETL
It has been extremely useful in Magic ETL 2.0 to have the Add Formula Rule available in certain tiles such as the Filter tile or Group By tile. I would like to see it be available in the Join tile as well. This would make it operate more like a SQL join. For example, if I need to use a BETWEEN statement for my join key, I…
-
JOIN on LIKE and BETWEEN in Magic 2.0
related to: https://dojo.domo.com/main/discussion/comment/28356#Comment_28356 It would be ideal to be able to perform a JOIN on a LIKE, BETWEEN, LESS THAN (or equal to) and GREATER THAN (or equal to) in Magic 2.0. Currently, users have to either do a full cartesian product (see threads above) or shift their pipeline over…
-
Magic ETL - Copy Single Tile
I share a lot of ETL code both internally and here on the Dojo. It's great that you can multi-select tiles in Magic ETL and select Copy to Clipboard, pass it along and have someone them copy and paste that into their ETL to get the same tiles. There are many times where I'm showing an example for a single tile in magic ETL…
-
Magic ETL Organization Tiles
Would be nice to have a feature to add a box around several tiles in order to organise better the dataflow. Sometimes we have a complex dataflow with several business logic and a lot of tiles, and it would be easier to identify the several steps with that boxes. An example of that is like the one in Unreal Engine:
-
Comment out tiles in MagicETL
When building complex ETLs it would be handy to be able to "comment out" tiles so they are ignored for execution. For bonus points, I'd like to be able to have an unterminated chain of commented-out tiles not block the ETL's execution.
-
More Dataflow suggestion: When I "cancel" out of a data flow don't dump me back to the Dataflow page
It doesn't always happen, but when it does, UGH. I hit cancel when in an ETL and then I go all the way back to the All Dataflows page. I don't want to go that far back. That's not even where I started. It seems to depend on how I get to the ETL. In this case I can be in the Overview page, click edit etl, once in the etl I…
-
Display Row count at every step in Magic ETL
Currently if you want to know your row count in Magic ETL at any stage other than the out put, you have to place a few temporary tiles to get there. I.E. add a rank and window tile to establish a Row Number column then aggregate with a group by to get the max row. Would really love if each tile could show the rows…
-
Undo button on cards and ETL
It is crazy that there is no undo button. There are countless times that I've had to go back to the beginning of something because I made an irreparable mistake in either a card or an ETL.
-
Turn on/off Magic ETL tiles
It would be really helpful to have the ability to turn on/off Magic ETL tiles. This is the equivalent of commenting out lines in a SQL dataflow so they don't execute when the dataflow runs but the effort in that section of the ETL is not lost and can easily be re-added.
-
SORT!
Need to be able to sort data from most recent date first to oldest date last for GSheet writebacks that take 22 minutes to finish so newest dates are available to view immediately while older dates eventually show up.
-
Alternate method to move a field in Select Column tile
In addition to dragging newly added field (column) to a new position in the Select Column tile, add an option to move the field by updating the field position number (e.g. from 150 to 30) similar to ordering columns in a SharePoint view. Dragging gets tedious when there are a lot of columns to traverse.
-
Add ORDER BY to GROUP_CONCAT Function in Magic ETL v2
Currently, Magic ETL v2 supports the aggregate function GROUP_CONCAT in its Group By tile. While clauses inside the GROUP_CONCAT function like DISTINCT and SEPARATOR are available, the ORDER BY clause doesn't work and causes the tile to fail. SQL DataFlows currently support using the GROUP_CONCAT function with the ORDER BY…
-
Data Science Access/Grant for Users in Magic ETL
We have users who use Magic ETL, but as admins we have users that should not have access to the features in the data science add-on. If there was a way to add a grant so we can revoke access to certain add-ons for users, this would be quite helpful.
-
Dataset reference in Python by identifier instead of name
To identify input dataset by identifier instead of name, very useful for working with Python modules, as right now you need to update the source code when changing a name.
-
Data Flows - Testing Use Case Filters
Use Case: When building data flows with data sets in millions of rows, it's hard to test if the logic works across those rows unless I add filter tiles that filter out data at specific points during the Data Flow build process. This can add complexity when adding many test cases at specific points. The Idea: It would be…
-
Make MagicETL Dataflow Formula Preview Match Data Output
in some cases when running the preview I get one result and when I run the dataset I get another. For example I have a formula SHIPDATETIME defined as " ADDTIME(DATE(`SHIPDATE`), TIME(`SHIPTIME`)) ". When I run this in the Dataflow preview I get the expected result when I run the dataflow the output dataset does not match…
-
Magic ETL: Avoid loading all the input columns and rows from data vault.
In Magic ETL, to avoid loading all the input columns and rows from the data vault. On the Input tile, under “Data Handling Section” if there could be a provision (refer to attached pdf with screenshot) check box to select required columns (simple request) and text box to add a where clause on each column Or one text box to…
-
Snap to Grid functionality in Magic ETL
Aligning the tiles in magic ETL can become quite cumbersome without any grid or guides for the tiles to line up with. It would be great for the default behavior of tiles to snap to a grid or show guide-lines that signify when tiles are aligned.
-
Nudge tiles in magic etl
It would be nice to be able to nudge tiles in Magic ETL for quick, easy, precise alignment. Thanks!
-
"Documentation Mode" in Magic ETL/SQL
A toggle next to "Try the new Magic ETL" would read "Documentation Mode" and would: Put an opaque film over the whole Magic ETL map Disallow changes to the tiles Enable the ability to draw and color-code sections of the ETL Be able to notate what each section is doing An additional item in the Save menu would be "Save and…
-
Magic ETL Append Tile - show full list of columns added to each input
In the Append tile, it shows you the count of how many columns will be added to each input (columns present in other input(s) that aren't in that one input). When you hover over the number, it lists the first 20 columns. It would be nice if it would display the entire list of columns and not just the top 20. When combining…
-
Magic ETL Creation from Card Filters Applied
I'd like to see the ability to create/save magic etl's straight from a card that has been filtered. This will make life simpler for magic etl manual creation.
-
Row counts based on node flow
Hello, I've been using Domo for over 3 years now and love the product. But there are multiple times when it would be nice to see row counts in each node along the data flow when you Preview the data. For example, please see attachment. This feature would allow someone to if data is being missed along the way due to a join,…
-
Support for having reusable ETL Logic
We are trying to redesign our existing ETLs (Magic ETL) whereby we want to have two identical sets of ETLs one for Full Load and another for incremental Load. Since the ETL is complex and there are number of ETLS that will adapt same pattern, we would like a functionality in Magic ETL where we can store entire Business…