-
ETL fails due to "This data set does not support data version filtering."
I have my SQL raw dataset set to merge new rows which is working correctly: This SQL dataset is the input for my ETL attempting to process New rows then Upsert in the output dataset: However my ETL fails to run due to the stated error: Am I setting something up wrong? My raw SQL data is not partitioned but it is using the…
-
Magic ETL Column Search Improvements
The new Column Search feature is a godsend. But … it could be better with two modifications: Add a Match String checkbox to limit the search to match the text string exactly. I was searching for the source of a field in a very large ETL. It turned out that the name of the one I wanted (countAsCar, for example) is included…
-
Data Join Confusion - Magic ETL
Hi, I'm trying to join two sets of data together (assignments submission data & assignment summary data) For the output, I am trying to get a list of all assignments with a due date in a date range that have not been submitted by the students. The assignment submission data has a row for every student that lists every…
-
Failed to run dataset due to error from the bery beginning node
I have a dataset that failed to run due to an error from the very beginning node. It said "Failed to convert value '5785586.012810127' from type 'String' to type 'Date'." However, that input itself runs normally and the data looks legit. I checked, no abnormal and no value of 5785586 anywhere. I tried to change all…
-
How to Filter for "Today AND Last 7 Days" in Domo Visualizer?
Hi everyone, I’m trying to implement a date range in Domo (using `Reservation Start Date`) that includes both Today AND the Last 7 Days at the same time. This will be for a weekly scheduled send. I know Domo has built-in filters for "Last 7 Days" and "Today," but I can't find a way to combine them in a single filter within…
-
How to calculate Months between two dates with DATEDIFF?
I was trying to use the formula DATEDIFF(month,date1,date2) to calculate the months between two dates based on the function details that shows up in Magic ETL (which I have pasted below). In the Unit form, the difference is returned in the specified units from the first argument until the second. For example, DATEDIFF(day,…
-
How should I change my calculation in the Filter Rows Tile in ETL?
Hi, I am using the Filter Row Tile in the Magic ETL. I want to filter the Overall Credits Column and only get the maximum for each student. However, the MAX and FIXED BY don't work. How should I fix this? I also tried the Add Formula Tile and it did not work either. MAX(Overall Credits)FIXED(BY Student ID)
-
Python Tile fillna on integer column keep return 0 unless I change the column to text
I have a column Bill-To (interger) I use Python Tile to run this script to fill the nulls cell with the previous cell value #Import the domomagic package into the script from domomagic import * #read data from inputs into a data frame input1 = read_dataframe('Select Columns') #write your script here input1['Bill-To'] =…
-
Do I have the right code in my Python Script?
Hi, Select Columns 2 has four columns, they are EFFDT, ACAD_PLAN, DESCR, and ACAD_CAREER. I wrote a Python script to filter the EFFDT column so that it only includes rows where the value is 'A'. However, when I try to join this dataset with another, I receive an error stating that the referenced column, which is the…
-
Why is it creating duplicate rows?
Hi, my first dataset has 21,683 rows, and the second dataset has 323 rows. However, when I joined them together, the resulting dataset has 31,045 rows. I noticed that there are duplicate rows. How can I fix this? Thank you!
-
Display Row count at every step in Magic ETL
Currently if you want to know your row count in Magic ETL at any stage other than the out put, you have to place a few temporary tiles to get there. I.E. add a rank and window tile to establish a Row Number column then aggregate with a group by to get the max row. Would really love if each tile could show the rows…
-
Display Record Count in Magic ETL on each step
Tl;dr - display the record count in MagicETL on each step of the last successful run of the ETL. Detail: I often spend quite some time diagnosing and fixing ETLs. In larger ETLs, it can be very onerous to determine whether a record with an expected behaviour makes it through the entire flow to the end to validate the ETL.…
-
Join in ETL
Question: I have two separate databases where there is only one common column, which is the "salesperson." The first database contains sales values for each salesperson. The second database contains the area of operation and products associated with each salesperson. I plan to create a new column to calculate the…
-
How do I add the current date to new rows that get appended by a recursive dataflow?
Hello, currently I have the standard recursive flow set up in the magic ETL, which tile and where in the recursive dataflow should I add to get the current date of when a new row gets added to the dataset?
-
Database Management View Instead of Datasets/ETLs view
Currently users view their assets in a list in the Data tab. They have to use naming conventions, or tags to group these assets and the same goes for ETLs. The list can be filtered and includes shared assets by default but regardless it quickly becomes a mess that only encourages redundant asset creation due to low…
-
Notes Tile in ETL
I know that we can make notes on every tile, but what I would love to see is a note tile where all it is, is a note. Sometimes I want to make generic notes in the ETL that relate to a few steps, or an overall process, or something relating to the business question not the tile specifically. So a notes tile would be great.…
-
Need help with specific code that is not working in either beast mode or magic etl formula tile
Hi the following code I am using as a filter on my card and have it set to "on list" its called skus to be purged "CASE WHEN On Order = 0 AND (Future Demand = '' OR Future Demand IS NULL) AND (Future Books = '' OR Future Books IS NULL) AND (Reserved + WMAVAIL <= 10 AND WMAVAIL > 0) AND WAREHOUSE_CODE = 11 THEN 'On List'…
-
How do I add a timestamp to the new rows that are coming in through my recursive magic etl?
I currently have a recursive magic etl set up to 'screenshot' data that is different from the source data. This information has been helpful because we can see the historical data any time a row in a column is changed. Like keeping a data log of all changes. However, I need to add a timestamp each time the recursive magic…
-
Join Data in ETL
I have two datasets: one for home country and another for citizenship. Both datasets share two common columns: country and student_id. In the home country dataset, the country column is 70% populated, leaving 30% of the rows as null. I want to fill these null values in the home country dataset using the corresponding…
-
When a value in column B appears multiple times against a value in Column A concatenate those values
Hi I'm really struggling with my dataset as I have a many to one relationship between the DIVISION and CUSTOMER columns which causes issue with grouping later in the ETL How do I write a formula which will affix the Customer name with the Division name if the customer appears against 2 or more Divisions else if the…
-
A Way to Group ETL Tiles Together and Collapse Them Similar to Subtotal Rows in Table Cards
Something I could see being beneficial, at least from a quality-of-life standpoint, would be a way to group individual tiles together in ETL that are related to a specific action/process that could then be collapsed into one larger "process" (or whatever it would be called) tile. This process tile could then be named based…
-
Appends Should Default to Include All Rows
Appends should default to "Include All Rows" and not change every time inputs are added/removed. Huge quality of life upgrade!
-
Magic ETL - Allow group by without a grouping column
There are times when I want to calculate aggregated metrics across the entire dataset. Currently this isn't possible without adding a constant value column to group on which I then have to remove later in my pipeline. It'd be much smoother if the Group By tile allowed us to group across the entire dataset without having to…
-
Pass Logged In User to Beast Mode?
Hi all, I am looking for a way to show total sales performance for various items relative to individual users performance for their own sales performance. Please see attached image for a better illustration of what I am trying to accomplish. In this example, I want to pass the logged in user name into a case statement.…
-
ETL Process to Analyze Google Reviews and Generate Automated Replies
Hi everyone, I’m exploring adding a step to an ETL process to analyze Google review data and generate recommended responses using ChatGPT or a similar tool. Currently, I have a Zapier connection set up, but it’s unreliable and inconsistent. Here’s what I’m looking to achieve: Dataset Input: I have a dataset of Google…
-
Is there any way to bring/read PDF files in their original format to domo from SharePoint folder?
I have a use case where I have to bring folder access from sharepoint to domo. When I access them they get converted to other file types. However I want the access to be in PDF format only and later would like to run a python script in Magic ETL to extract specific data from pdfs to update rows in Microsoft excel. Is there…
-
How can I set up a graph to show period over period data in a dynamic way?
I am looking for a way to show a bar graph with sales data by period and a growth rate line with period over period percent change. I have a detailed dataset with a large amount of sales data spanning over multiple years. I am having trouble with how to get the prior period data to be on the same line and to be dynamic so…
-
Multiple results encountered for the same location in Pivot Table indicated by ****
Attached is a sample of my data. I am trying to create a pivot table with the following: Rows: location, week, date, shift Columns: count(unique identifier), count(unique identifier) * shift length In my ETL, I created a formula to handle empty values. It determines whether a shift was 8 hours or 10 hours, based on a…
-
How to shift first week of year from December to January?
The card pictured below is filtered to 2025 - however, Domo's default week starts the prior Sunday which still lands in the month of December. I would like it to show a January date instead for the first week of 2025. I would ideally not like to use week numbers over actual dates on the x-axis. Has anyone else experienced…
-
How to create WoW, MoM, QoQ, YoY dimensions to put into variable?
I want to create a dimensional variable that allows users to choose between WoW, MoM, QoQ, and YoY using the chart types below. What is the best way to go about this? Any tips on creating the ETL to create the components above?