-
Creating a Cumulative Concat Function on a Text Field
I currently have an ETL looking at distinct Opportunities by their close won day. However, this issue is now looking at Distinct accounts within the same dataset. At first, I thought to use the count distinct function on the account ID in a groupby. However, since multiple opps can be tied to one account and each one can…
-
Conditionally Starting a Dataflow
I have a dataflow that I only want to start when the data in a dataset materially changes. It seems like it runs no matter if the data changes or not. Any advice on this.
-
Options for Preview Run Limits in Magic ETL
The Magic ETL currently only max out at 400k row for the preview run from your input datasets. This is limiting the test preview in your flow as it only picks (from my understanding) the first 400k row of your dataset. So if you're trying to preview your actions (such as filtering) you may not get anything showing in the…
-
Magic ETL Checklist Enhancement Idea
Hello, It would be helpful if we had the option to add items to the Magic ETL Checklist. This would allow for each organization to be able to make sure everyone is designing their ETLs per their procedure/guidelines. For example, if we are requiring the designers to 'label' all tiles, then we could add a checklist item;…
-
Pivot Data in ETL
I have dataset i am trying to reformat using a pivot table. I cant use the pivot chart because it only shows measures as values so i am trying to reorganize my data using pivot in etl. My data includes Facility, Date, and Task Completed (yes/no). Beow on left is how my data currently is and on right is how I want to set it…
-
ETL error: "failed to convert value 'Infinity' from type 'Floating Decimal' to type 'Fixed Decimal'"
Hi, I have a magic ETL dataflow. It has 2 input sources. There's a formula tile where we calculate a pacing percentage. (`Delivered Impressions` / (`Campaign Flight` - `Days Remaining`)) / (`Total Goal` / `Campaign Flight`) We've been using this dataflow and formula without any issue for months and now we're getting this…
-
Custom Calendar Week On Dataset?
Hey! What if I want to establish a new calendar on a dataset? For instance, the third week of January would be CW1 and each week starts on a Saturday. thanks!
-
Applying Filter Tile to Remove Nulls Not Working
Any ideas as to why this isn't happening?
-
Show "Output Rows" by Action in ETL run details
Currently when you view the run details of a MagicETL dataflow, we can see the rows processed, duration, and status of each action in the ETL canvas. It would be very helpful to see an additional column that shows the number of output rows of each step, similar to how we can see the output of each transform in MySQL…
-
Multiple case statements
I have 3 columns in a table with with fruit name , benchmark value and achieved benchmark value. Ideally I would like to use colors on a table when (in this example) : Apple is equal or greater than Apple benchmark value then Green, else Red Orange is equal or greater than Orange benchmark value then Green, else Red Pear…
-
Ability to remove unnecessary columns directly in the input DataSet before loading all its rows
The functionality I require is already available in the MySQL and Redshift DataFlow… …in this example deselecting "department" and "state" columns. This option would make it possible not to load all columns for all rows in the DataSet. I hope it will be available in the Magic ETL input DataSets as soon as possible. Thank…
-
Find and append a value to a field
I'm trying to align some inconsistencies with our data. What I'd like to do is find any value that does not end with a '/' and append it with that character. Is there a case statement where we can look for a field that does or does not end in a character? I assume if I can do that, I can just append those without with the…
-
Count of Active Employees Month by Month
I have a card to build which should show running graph of how many candidates had an active authorization at a particular office locations during a given month. It should be able to show historical running totals of employees that an office has and are they growing or shrinking. So, I have dataset which has columns like…
-
Prioritize one set of data over a duplicate set of data, when available.
I have similar data coming in from two different sources. I need to make sure that when dataset A is available, it is used, and when A is not available, B is used. I want to end up with one row of data per day per website, but I either end up with only one day's worth of data per website (Hostname in my data), or I end up…
-
Having Multiple Group By tiles?
I am looking to have two separate groupings in one ETL. Is there a way to do this without having to use two Group By tiles? Also I no longer see the minimum aggregate option for my new column. Do I now have to use the Min() formula?
-
New Dataflow Run Options
Looks like Domo has silently updated trigger options for dataflows (hopefully its not silent and they tell people, I think it just got to my instance about an hour ago so hoping I am just early and more details will come out soon). I love this idea but do feel it is a bit clunky and requires a lot more clicks but I need to…
-
ETL Field Search Capability
It would be helpful in an ETL to be able to search for a field name and have the tiles that directly reference that field name be highlighted. Many times when supporting dataflow development we are looking for a field and we have to open every tile to see where that field is referenced - in join, in formulas, in filters,…
-
Need Rank to update in non-standard (?) way
I've got customer records with customer GUIDs, record dates, and member status. If the data for a specific customer looks like this: Date Status 1/1/2023 Active 2/1/2023 Active 3/1/2023 Inactive 4/1/2023 Inactive 5/1/2023 Active 6/1/2023 Active 7/1/2023 Active I want to get a dense ranking that groups the status together,…
-
Appends Should Default to Include All Rows
Appends should default to "Include All Rows" and not change every time inputs are added/removed. Huge quality of life upgrade!
-
Issues with changing cast dataset to date
Hi! I am trying to change the data created column to date, but when I check append rows, it says the column is changing to timestamp. I am not sure what I am doing wrong.
-
routing ETL failures through zapier
Is it possible to send ETL failure messages to Slack using Zapier or a similar service?
-
WEEKOFYEAR is not aligning to ISO-8601, is this correct?
Hi, I recently utilised the Week of Year function for a date column. I'm unsure if this is incrementing from 0, but the value for the 12th of Feb 2024 returns 6. I would expect it to return 7. . Can someone confirm if this is intended? It looks to use Sunday as the start of the week in this function. Best, Sam
-
MAGIC ETL/DATAFLOWS Q&A from Domopalooza
The product team enjoyed answering questions from the audience during the final session at Domopalooza. Below you will find the answers to all questions related to Magic ETL and Dataflows (including those that weren't mentioned on stage). Q: When are you going to upgrade the MySQL version in MySQL DataFlows? A: We do not…
-
Group By issue
I am doing a group by tile in magic etl. i have different values for different ID's and need to sum up all the values in each ID. For some reason some of the ID's are not summing and I get null in column for the ID. I did formula to make sure all null values before grouping are 0 thinking that would fix it but for some…
-
Allow users to specify the color of Magic ETL tiles
This would be a small change that could help organize extremely busy dataflows. While you can currently change the color of the line
-
Show Configuration AND Notes panels on Magic ETL tile simultaneously
Currently, a user can see either the configuration of an ETL tile OR they can tab over to the notes section. It would be nice to see both views at once when writing notes on the configuration or editing the configuration based on the existing notes.
-
Reloading Recursive Dataflow Data
Hello, I have a recursive data flow running daily adding on and updating our sales data. It was discovered that the join wasn't using a fully unique identifier so some rows of data have been cut out over time. I want to clear out the data and resend it all to domo so that I make sure I have it all. Is there a way to clear…
-
Can I see the basic dataset length between different steps of an etl flow?
For example, I want to know how many rows are in the table before and after a pivot.
-
Preview Window Limits in Magic ETL
There is currently a 100 rows preview window limit when you do the preview run on the ETL. We need to increase this as it can help drastically with further work in the MagicETL.
-
Maintain Input DataSets when Pasting Copied Magic ETL Tiles
Currently, when copy and pasting Magic ETL tiles between flows, input tiles are scrubbed and become a plain input tile without a DataSet selected. I understand why you'd want to scrub this if pasted in the same flow, but it should be maintained when pasting in a different flow. This should be relatively easy to implement…