-
Group By Question
Im trying to do a group by to get a new column with total sum of charges by bill ID. I want the total sum to be constant all for every day associated with Bill ID. When I do a group by I am not getting the new column to sum the charges. Bill ID Date Charge 502 4/10/2023 $85 502 4/11/2023 $76 502 4/12/2023 $110 502…
-
Allow Window Functions in MagicETL Formula Tile
It would be helpful to be able to write MagicETL formulas using window functions, similar to in beast mode. While the Rank & Window tile is often sufficient, there are some use cases where having this feature in the Formula tile would simplify the dataflow steps: Using a sum window function with unbounded rows This does…
-
Block tiles in Magic ETL to simplify complex dataflows
"Block Tiles" are larger tiles that contain a string of tiles inside. Block tiles could make large Magic ETLs (100+ tiles) easier to document, and much easier to read and understand. These tiles would contain a group of tiles that have a single output tile (not an output dataset) to make for an easy way to connect block…
-
Magic ETL - Allow group by without a grouping column
There are times when I want to calculate aggregated metrics across the entire dataset. Currently this isn't possible without adding a constant value column to group on which I then have to remove later in my pipeline. It'd be much smoother if the Group By tile allowed us to group across the entire dataset without having to…
-
Assign colors to tiles
I would like to be assign colors to tiles: this would make troubleshooting and code reviews much easier. A stretch goal would be to add setting the color of tiles to the options when you select more than one tile in addition to Delete, Duplicate, and Copy To Clipboard. Using this to visually indicate functional sections of…
-
Add Formula Rule to Join Tile in ETL
It has been extremely useful in Magic ETL 2.0 to have the Add Formula Rule available in certain tiles such as the Filter tile or Group By tile. I would like to see it be available in the Join tile as well. This would make it operate more like a SQL join. For example, if I need to use a BETWEEN statement for my join key, I…
-
Magic ETL Organization Tiles
Would be nice to have a feature to add a box around several tiles in order to organise better the dataflow. Sometimes we have a complex dataflow with several business logic and a lot of tiles, and it would be easier to identify the several steps with that boxes. An example of that is like the one in Unreal Engine:
-
Comment out tiles in MagicETL
When building complex ETLs it would be handy to be able to "comment out" tiles so they are ignored for execution. For bonus points, I'd like to be able to have an unterminated chain of commented-out tiles not block the ETL's execution.
-
More Dataflow suggestion: When I "cancel" out of a data flow don't dump me back to the Dataflow page
It doesn't always happen, but when it does, UGH. I hit cancel when in an ETL and then I go all the way back to the All Dataflows page. I don't want to go that far back. That's not even where I started. It seems to depend on how I get to the ETL. In this case I can be in the Overview page, click edit etl, once in the etl I…
-
I need to make a bar chart that combines multiple columns based on date
Simply put, I want to display how many users completed each program in Q4 2023, and I want to later be able to select a different date (so later, for example, I can see how many people completed programs in Q1 2024). I want my users to see the most popular progams during a given time period. I think a bar graph will…
-
Different time zone
Hi! I'm trying to configure the time zone in ETL but my query is resulting in different times when the function calculates the difference, for the same column. ETL time zone function: Imported table from SQL Server: Result after time zone changes: The date range of the table is from Feb 24 to Mar 12. From Feb 24 to Mar 8…
-
Nested Case and Concat Statement
Hi, I am trying to figure out how to create an Etl that expresses: If a particular year has a result then concat that with another column's result. Something like: CONCAT(case WHEN 2015= 1 then '2015', '-', Virtual vs In-Person else '' end) 2015 has a 1/0 constant and I want the result to be if they attended in person (so…
-
Is it possible to batch edit source data?
I have a large dataset that needs some cleaning. We have appended to this dataset daily over several years and some of the data values have drifted over time as users have renamed what are essentially labels for different entities. If this were SQL, I could write an update statement and quickly replace the values in label…
-
how to combine mutiple rows into one separated by ','? Form builder muti selection data handling
I used form builder and there are couple fields are multisection dropdowns, in the response dataset, it separates the selections into different rows, but I want them to show as a,b,c together not a(next row), b(next row),c. Does anybody know how should I use the magic ETL to do that? I used my old way to transform the…
-
Renaming Columns and NOT breaking cards
I am barely getting started with the vast DOMO world so forgive me if this is an obvious thing. I was recently creating a dataflow and some cards from that dataflow and decided to test something out. I created the card using a column from that dataflow and then proceeded to change the name of that column in the dataflow.…
-
ETL calculation to work out hours taken from a start and finish timestamp
Hi all and happy Monday I have some data i am pulling in via a SQL workbench that gives us a time clocked onto a job and another column for clocked off the job. The data is in a the format 2022-12-23T14:27:20. So i am trying to generate a calculated column that gives me the hours taken per job, for example: Clock on =…
-
Lag a pre-existing column?
Hi there, I'm wondering if it's possible (with Magic ETL or beast mode) to creation functions (like rank and window) that can lag value from Column 1 to Column 2, where both columns already exist. Full context: We have 3 columns. Monthly total, Last month, Year to date. I'm trying to generate a new YTD total every month by…
-
How do i fill in rows of data with previous row info?
This… into this via magic etl
-
When alter columns can't change column type, what else can I try?
-
Joining row data
Hi, I'm trying to combine several row data that have a similar dimension. For instance, i want to aggregate all of the lists_name's into one row next to 2024 Collection - Launch #4. so turn this… into this…
-
Trying to convert Month of the Year into Date with Magic ETL
Basically the title. I am using a dataset that pulls Month of the Year instead of Date, as the User count with the Date metric isn't accurate. However I have run into issues trying to convert Month of the Year into a date format. Here is the original format: As you can see, each month is numbered from 1-12. I used Value…
-
Converting to Integer
Hi All, In my ETL I am loading 2 columns for Credit & Debit, as part of my ETL I want to do Credit-Debit to get a column for amount however the data being loaded is recognising the Credit & Debit as text. I've tried to use the alter columns tile in my ETL to convert to an integer however it doesn't appear to be working.…
-
Turn on/off Magic ETL tiles
It would be really helpful to have the ability to turn on/off Magic ETL tiles. This is the equivalent of commenting out lines in a SQL dataflow so they don't execute when the dataflow runs but the effort in that section of the ETL is not lost and can easily be re-added.
-
Keep Columns After Using Group By Tile in ETL
I haven't played around with the group by tile yet so this might be a very simple answer I overlooked. But why do all of my columns disappear and leave only the columns I identified in the group by tile from my data after I use this function? I know I could join the data back in but I was assuming I was using it wrong.…
-
Run Recursive Dataflow only when new data comes
I am trying to create a recursive dataflow which should run only when new data comes. Here are the conditions for that, We are suppose to get weekly data in our CSV connector every Monday. We have set out recursive dataflow to run on Wednesday. We use week number as our constraint column. But sometimes, we get data late on…
-
SQL Query against dataset
Hi, I am wondering if anyone has ever had a ask for a way to just write sql against a dataset, or join to datasets. I know we have redshift, but this would be more like a SQL IDE you would use in a relational database. I know its kinda an odd question, but we have some users who prefer sql vs magic etl. I have seen the AI…
-
Date Conversion
I need a formula for taking just the month name from this format - Thu, 29 Feb 2024 11:59 PM. I need it in a date formatted column though. I can take the Month Name using MONTHNAME('my date column') but i cant get the column to be date. Its always text. Any advice? Thanks
-
Required Role Level for Reverting Dataset History
Hello, Domo Community! I'm in the process of delegating dataset management tasks and need clarity on permission levels. Specifically, I want to understand the minimum role level required for a user to access the history of a dataset and use the "revert to this point" feature. I've developed some reports and plan to allow…
-
No Idea why I can't get string to datetime
Input into formula tool is 2019-01-02 07:30:00 AM, formula is STR_TO_DATE(TimeIn1,'%Y-%m-%d %h:%i:%S %p'). Validate formula checks out, but error is Failed to parse date/time: Error at or near character 11 in string /2019-01-15 7:30:00 AM