-
Domo ETL parallel processing to reduce ETL runtime
Hello, I have a python script which is part of an ETL to transform a combination of 3 input datasets. The python script contains multiple if else statements and for loops which is taking aroung 4 hours to complete the ETL pipeline. I want to know if there is an option to parallelize the python script in Magic ETL to reduce…
-
Data transformation ideas around old/new user data
HI, we're migrating from username sign in, to okta that uses email as user id. we have an existing user info data with user id as username (xfsgsbss) but we would have a new data with user id as email(fdhsvbsjsbsjsh@email.com). I want to be able to join both old and new data and make sure the right attributes goes to them,…
-
How to create a rate by specific selection criteria in Domo
Hello again community. So I work within a healthcare organization and I have a domo grouped/stacked bar graph built currently from an etl that has columns regarding patients readmission start date, account number, discharge date and attending providers(previous admission). The card breaks down the readmit percentage rate…
-
Using Magic ETL to extract a substring using Regex and excluding remaining strings
Hello! I know variations of this question have been asked in the forums but I haven't been able to find a solution that works for me. I have a column Project Name that contains Projects out of Financial Force that may contain an At-Risk ticket number, which is formatted as: AR-00000 i.e. literally "AR-" or "AR - " followed…
-
More info in logs
It would be very useful if, when executing a Domo ETL, a log file was generated to see the details of potential failures in the process. Currently, if there is a failure in any of the Domo's own boxes, you see that the process has been interrupted in a part of the process but we don't see the details. Also, if the process…
-
Saving States
Is there a way to save data and possibly append to it? Let me give an example. When I pull data from a database, I would like to pull data and when I pull data tomorrow I want the original data to still be there. So I want to be able to see user counts by region today, and then in a month I would still like to see what it…
-
Joinin on Email
I have 2 contact databases where I need to join them based on both phone number, name, and email. I am able to currently join on phone but the email part I am having an issue with. When trying to join on email it isn't working. What is the best way to join on email address and if possible where emails closely match?
-
Split Column by Number of Characters
Can we add an option to split a column by the number of characters in the field? I.e. split column by number of characters as far left or as far right. This is a very useful feature in PBI Power Query Editor especially when you have strange date formats like 1231201 or concatenated fields with an order number of a specific…
-
Rank Tile Not Grouping, Needs Case Sensitive box
I am using the Rank tile, ordering on a Timestamp and the column that makes up a partition is Customer. For example, I have "CUSTOMER A" and "Customer A." These are in the raw data and represent different clients, but the Rank is treating them as the same. I will probably value map one of variations to be unique, and I'll…
-
Rank and Window? Lag?
So I have a dataset that captures quarterly expense data. I have been tasked with adding an 'expense ratio change column' Basically, I need to know the % change of expenses changed from quarter to quarter by project I know the math on this ((p2 - p1) / (p1)) but am not sure how to tile it in domo to make sure it is taking…
-
How to mix and match Data
Number City 1000 A 1100 B 1200 C 1300 D E 1400 1500 F 1300 Hi Folks, Need your help pls. 2 Questions - 1) In the table above, you could see that 1300 is mapped to city 'D' but in last row it is blank. How do I map it using beast mode or Magic ETL to fill by Lookup in the same column. Req: Lookup in 'Number' and fill Blanks…
-
Does a slicer work even if I'm using a sub dataset?
Hello. Hope someone can enlighten me or help me if there's a solution for this: I need to use one slicer for my dashboard, which contains like 16 cards. The slicer should be able to filter all cards in that dashboard. Here's my problem: I'm using three datasets for the cards in the dashboard: Main Dataset - let's name this…
-
Is there a way to duplicate a value one row down?
I have a column with a numeric value and null value every other cell. I want to replace the null cells with the numeric value that is directly above it. Thanks in advance!
-
Weighted Average Calculation
Hi, I'm working with a dataset and am wanting to make sure I am going to get a weighted average of our mortgage rates. We have loans that range from 100k-400k and each loan has an associated mortgage rate. As an example, we have two loans that go into the same security that are 5% and 5.5% Loan A is worth 200k and loan B…
-
Where can I find the Description when you save a dataflow
I would like to create a card showing the changes to dataflows and the description. I have looked in domo stats and also domo governance and dont see it. I am looking for the comment you add before you save. Not the description of the dataflow that you put along with the title.
-
Add Notes and Titled Containers to ETLs
I find that my ETLs are often big, with multiple endpoints; so when I come back it takes me some time to catch up with what I did. I think that adding some text notes to the ETL view would make it easier to evaluate what I, or one of my coworkers, have done. As for titled containers, they would allow a user to draw a…
-
Combine tables using ETL inner join
Hi everyone, I'm trying to combine two federated tables using the inner join in Magic ETL. Both tables have over 9.5M rows but when I join them, the result is only 87K. I tested the join in Snowflake and it resulted in 9.5M by the logic: SELECT * FROM 'A' INNER JOIN 'B' ON A.X = B.X What could i be doing wrong? *I'll need…
-
Add Dataflow Comment to Domo Stats | Domo Governance
I would like to create a card showing the changes to dataflows and the description. I have looked in domo stats and also domo governance and don't see it. I am looking for the comment you add before you save. Not the description of the dataflow that you put along with the title.
-
Append Output dataset help
Hello all, I have an Magic ETL (A) that with a output dataset - say OUTPUT_1. What I am looking for, is when every time I run the ETL, I need the output results to be appended. At the same time it should not create duplicate records. I created a new Magic ETL (B) with this OUTPUT_1 as input dataset and outputted to…
-
Question on Domo SQL syntax
Hello all, I am trying to do Magic Transform using SQL option. I have embedded a SQL query but it is erring as it is not compatible with Domo syntax. Appreciate your help in this regard. DATE_PART('WEEK',CAST(TO_CHAR(TO_DATE(TO_CHAR(PRODUCT_KEY),'YYYYMMDD'),'YYYY-MM-DD') AS DATE)) AS "SaleWeek",…
-
Need help mimicking recursion with partitioned datasets
I need help implementing a particular use case for partitions: importing selected partitions from a dataset, modifying the records, then outputting the data back into it's original partition (or a new partition) in the original dataset. There are two uses for this I can think of off the bat: A dataset can have a maximum of…
-
Aggregate and ETL question
Hello all, I have the below workflow in my Magic ETL. Just posting a sample version below. My question is - I have a ProductID that I join from both the data sources and then do a group by and output to a file. But I also need to know Total count of Product ID or row count from TEST spreadsheet source. Is there a way I…
-
How to count status changes in snapshot data
I have data that I snapshot daily. It has employee name and employee status. Once the status gets updated, it remains that way: ex for one employee: 1/1 Name: active 1/2 Name: active 1/3 Name: active 1/4 Name: Inactive 1/5 Name: Inactive 1/6 Name: Inactive 1/7: Name: Inactive At the card level, I would like to be able to…
-
FIXED BY Beastmode trouble shooting
Something is not working quite right and I am having a hard time figuring out what is going wrong. We're trying to see the % of registrations by year, by day of week (sun, mon, tue, …), by hour of day, and then to be able to filter by locations. So in ETL, for the numerator, I am grouping by Year, DayOfWeek, Hour of day,…
-
How to input identifiers using identifiers
I have a data flow that has multiple properties and competitors. I was looking for a way to label all of the records that are in between into one property. In the example sheet that i uploaded, I was wondering if a beast mode formula could be used to rows 3-16 are labeled as the property Brandon crossroads in this case.…
-
Sum column when i have duplicates in other two columns?
I have these columns and I would like to sum the cost column only when the Publication and Year column have duplicates, so I can see the cost per Publication per year. Brain fart, I am assuming this is pretty straight forward? I'd like to do it in ETL if possible. Is there some SQL I can utilize to do this in the formula…
-
What function will help me determine a variable part of text?
I'm using Magic ETL to create a new column that will return True or False depending on the contents of a string in another column. Question is how do I write a Case formula that will return True or False based on the contents of another column? For example, the word "Delivery" is placed in different locations on a string…
-
Date format question
Hello all, I have a snowflake i/p data source. One of a column is a date field and it showing up as floating decimal in Domo. This field has value -3 (even in data source) if it is a null else it is a date.. As this is floating decimal type in domo, under configuration I changed to Date type. When I try to join with…