-
Magic ETL Won't Save DataFlow?
Team, I have built an extensive ETL and I want to "Save and Run" to explore and analyze the output. Unfortunately, the "Save" button isn't responsive. I will click on it and it won't actually save. I'll click "Save and Run" and it won't actually save… Weird issue that I've never ran into before so hoping there's a simple…
-
Joins: Automatic Identification of Relationship Direction
Please add: Automatic identification of relationship direction within a Join tile. An option to amend this relationship direction (where appropriate) within the Join tile. E.G A drop-down with 1-2-1,1-2-Many, Many-2-1, Many-2-Many (inc warning) e.t.c A visible representation of this relationship direction with ERD arrows…
-
Hi I had a question about creating a view by joining two tables in the data tab of a card.
I am trying to join two tables to create a view in a DOMO card but I am running into a weird issue. Table 1 has 42.3 million rows and table 2 has 158K rows. When I do a left outer join to table 1 I am getting 297 million rows of data. Why am I getting more data than the two tables themselves it should be less data, not…
-
Data Join Confusion - Magic ETL
Hi, I'm trying to join two sets of data together (assignments submission data & assignment summary data) For the output, I am trying to get a list of all assignments with a due date in a date range that have not been submitted by the students. The assignment submission data has a row for every student that lists every…
-
Join in ETL
Question: I have two separate databases where there is only one common column, which is the "salesperson." The first database contains sales values for each salesperson. The second database contains the area of operation and products associated with each salesperson. I plan to create a new column to calculate the…
-
Creating a join expression with multiple criteria
Hi, I am a little stuck trying to create a join expression. In the table below, I want to join the Cost using the active From date to a table of data with multiple dates. For example, I have data from 1/31/24 thru today. For Campaign 1, I want to use the cost of .5 up to 1/31/2025 and then moving forward after that, use .6…
-
Join Data in ETL
I have two datasets: one for home country and another for citizenship. Both datasets share two common columns: country and student_id. In the home country dataset, the country column is 70% populated, leaving 30% of the rows as null. I want to fill these null values in the home country dataset using the corresponding…
-
DataDog Dataset Join to Showcase Monitors and Events Together
Hello, I am trying to join the DataDog Events and DataDog Monitors datasets but I have not found any way to join them. I want to show every event that has occurred for each monitor over time with their severity. I have been looking at each of the respective datasets IDs and I could not find a primary/foreign key pair to…
-
Transforming an ETL
Hello! I am trying to edit an ETL with three input datasets.. My goal is to get the 'GLAcct' row of data from the "GLAC PARTS (EZ TEST)' input dataset into the GLBD_V2 datset. Currently, the join is taking the 'GLAcct' under the input dataset 'GLBD_v2'. The request I received was to add a column onto one of our cards…
-
Group By Issue
I need assistance with identifying transactions that net out to zero in a large dataset. Each row in the dataset represents a unique transaction with its own ID. Objective: Identify pairs of transactions that meet the following conditions: Same client ID Same transaction type ID Same transaction date The sum of their…
-
Income Statement: How do I join 'Chart of Account' table & 'GL Records' correctly
I have 2 tables: Chart of Account - Right Table: notice that there are Addon Repairs, Restocking Charges, Contractor Discount in 'Cost of Sales Others' 2. GL Records (for all branches from Feb 2024 - Sep 2024) - Left Table Problem: I've tried both Full Join, Left Join (key: gl_account_id) then make a pivot table (filter a…
-
Logic - link original submit date to all subsequent back orders
Hello. I'm wondering if anyone has ever had a similar scenario/concept and have come up with good ways to achieve the desired outcome. I've attached a screen shot of real data. The goal is to get the submitted date of 7/5/2024 (A6) linked to orders in rows 7 and 8. The story of this data is that order id 16403748 was…
-
Selecting prefix for conflicting field names
When joining datasets, Domo alerts you to any conflicts in the naming. Specifically, any common field names have to be renamed. It defaults to using the name of the originating dataset as a prefix. We use longish, descriptive dataset names, and the narrow, static width of the editing line makes renaming very tedious. Is…
-
Magic ETL Join Tile Alterations Field Search
We have really appreciated all the quality-of-life updates to Magic ETL over the past few months! I have a few ideas to improve a pain point we frequently encounter when building ETL's. Some of our standard schema is verbose and requires field name prefixes when ingesting from certain systems. When we attempt to alter…
-
Budget to Actual - Multiple Dimensions & Different Datasets
I have a budget dataset with the following columns: _Account _Subsidiaries _Department _Doctor _Location (Not in Use) _Headcount Resources _Placeholder 4 _Year _Period _Scenario _Currency _Measure _value I have an actuals dataset with the following columns: Appointment Date Status Appointment Type Patient Account Number…
-
Add Formula Rule to Join Tile in ETL
It has been extremely useful in Magic ETL 2.0 to have the Add Formula Rule available in certain tiles such as the Filter tile or Group By tile. I would like to see it be available in the Join tile as well. This would make it operate more like a SQL join. For example, if I need to use a BETWEEN statement for my join key, I…
-
Joining and getting all dates between start and end date
Im doing a join on billing ID's. Each billing ID has a start date and an end date. Im trying to return all dates between start and end date for each billing ID. I tried adding this formula to join. dt BETWEEN Service Start Date and Service End Date It isn't working as expected. Im just getting one date for each billing ID
-
Combining two datasets based on two potential fields
Hi, I have two datasets. One has pacing data, the other has monthly data. All rows have unique IDs, however the pacing data may have either an ID from the old system (legacy ID) or the new system (new ID). Pacing: ID 1 2 3 4 11 Monthly: New ID Legacy ID 10 1 11 12 13 2 14 3 15 How would I go about joining these two…
-
Issues Joining Transaction Details to Primary Dataset
I have two datasets. One is structured as the "primary" expense file, where each row is dates and other dimensions for each credit/debit. The next is credit card transaction details. They share transaction IDs in a column named Doc, share GL #'s, and share posted dates. The combination of all 3 of these things will result…
-
Dataset Views - JOIN on date range?
In most database systems, you can create a join/view on date ranges like so: SELECT * FROM A1 JOIN A2 ON A1.ProductID = A2.ProductID AND (A2.SalesDate >= A1.StartDate) AND (A2.SalesDate <= A1.EndDate OR A1.EndDate IS NULL) However, a DataSet View in Domo only seems to allow joins on exact values and does not allow ranges…
-
Comparing Datasets to Determine Variance
Hello everyone, I am working on comparing data from a previous data input, Universal Analytics, to a new one, Google Analytics 4. Basically, there was an update and data is formatted differently. I am attempting to join the two datasets to compare the month of A ugust for variances, but the closest thing to a unique column…
-
Join Condition in ETL
1.For Some scenarios we will join tables based on the between dates condition with out columns mapping Eg select a.*,b.* from Table1 a , Table2 b where Date between date1 and date2 2.For Some scenarios we will join tables without any conditions Eg select a.*,b.* from Table1 a , Table2 b
-
How to exclude/filter out based on 2 dates
Hello Domo! I uploaded a file/table in DOMO, that has a wipe date and a forecast date. What I was hoping to do is, to use this table and join it in my dataset, so that I can use it to filter out everything within the wipe date and forecast date. Is this possible? I am having problems with joining it to my dataset.
-
Combine tables using ETL inner join
Hi everyone, I'm trying to combine two federated tables using the inner join in Magic ETL. Both tables have over 9.5M rows but when I join them, the result is only 87K. I tested the join in Snowflake and it resulted in 9.5M by the logic: SELECT * FROM 'A' INNER JOIN 'B' ON A.X = B.X What could i be doing wrong? *I'll need…
-
GA4 campaign and session campaign data Joining question
I have a unique question… some context: I have a dataset from the Google Analytics 4 connector that has 5 columns that I am grouping by with one of those group by columns being the 'campaign' dimension. The other dataset in the join has 4 of the same columns but the fifth one is a 'session campaign'. The two datasets are…
-
Joining 3 different datasets to match zip codes
Hi, I have 3 datasets each with a column containing zip codes. I want to use ETL to create a new dataset with 3 columns, all containing the zip codes from each input lined up. Input: Dataset 1 Dataset 2 Dataset 3 1 2 1 2 3 3 3 5 4 4 6 5 Output: D1 D2 D3 1 1 2 2 3 3 3 4 4 5 5 6 Thank you
-
Datasets join issue
I'm trying to create a card that can be used to present to budget owners how much they have currently spent for the month compared to the budget they've been given. In order to do so, I have to create a dataflow that involves joining 2 datasets; One is Netsuite, which stores all the journal entry transactions, and the…
-
How to apply a filter across 2 data sources with filter column having diff names in the data sources
I have a column called Frequency (values could be daily, weekly, monthly) in data source 1 and a column called Cadence (values could be daily, weekly, monthly) in data source 2. How can I join these two columns to apply the filter on both data sources?
-
Combining Rows of Data with a common attribute
Hi, I have a dataset which is product data cross multiple years. In some cases the same buyer bought the product in year 2022 and year 2021 and I want to roll this up into 1 line. How do I do this automatically, matching on the word 'Buyer C' (because it's not feasible to manually match them with the size of the data) Any…
-
MySQL Indexing Issues
When setting up Indexes in my SQL dataflow, I notice sometimes that a few columns in the join are not made available for indexing. Why is that? What can I do to fix this? I suspect without the indexing, my dataflow is taking an inordinate amount of time. Also, in the Domo help on optimizing data flows with indexing, the…