-
What are best practices for combining huge datasets in Domo?
We have been using Domo for nearly a year now and are heavily utilizing DataFusion and Dataset Views to accomplish joining dimension tables to our primary fact tables. By the time we have joined everything we need, it is not uncommon to be looking at 300-800 columns and 50-100 million rows across 5-6 datasets. We often see…
-
How to create a column which SUMS based on the grouping of another column
Hello, I am trying to write an ETL which sums a value based on the grouping in another column (invoice number) which yields a column with the total quantity for each invoice. i have attached a simple table as an illustration of what i am looking to achieve. I do not want to write a formula where i have to continually…
-
Why do all Redshift Datflows (still) convert all variables to lower case?
Hi, we're currently doing an audit and noticed that legacy datasets have 2 versions, with the 2nd version just capitalizing the variable names (often a magic ETL). The first version is in Redshift - building dataflows in redshift is quite popular within our team because it has fuctions that mysql dataflows doesn't have.…
-
Breaking down bundles into individual items
Hello Everyone, I am currently pulling in sales data from shopify. We have a core range of about 20 products and another 20 or so bundles which are made up of those core products. I am hoping for some assistance with how to break down these bundles into its components (core products) as we would love to see how the core…
-
What is the best way to calculate?
So I'm using a grouped bar graph and am looking to try and calculate the difference between two of my columns that were created. My company calculates the average spread on loans broken down by lenders we use. I was able to separate those average spreads by product by adding the products to the 'series' drop-in at the top.…
-
Attempting to create unique rows for each possible value in a chosen column
Our business model currently offers multiple product options. I'd like to create brand new rows for each customer, 1 for each product, whether they own that product or not. My thought is to use new columns and then use a dynamic unpivot. Need help creating the new columns first though.. Currently have this: Account Product…
-
How can we download list of beast mode made on particular Dataset ?
Like I can see the list of all beast mode on a particular dataset on beast mode manager .But I want to download them also as per the particular dataset . Suggest me some way if there is any .
-
Is there a better way to perform a variable-length recursive query using MySQL ver. 5.6?
I'm looking to perform a recursive query on a data hierarchy with an unknown amount of levels. I've found that making use of stored procedures (or simple self-joins if the there are very few levels) works well enough most of the time, but it can be messy and time-consuming. From what I understand, Common Table Expressions…
-
Domo instance API - Method Not Allowed
We want to interact with our Domo instance via scripting namely for triggering a Magic ETL or MySQL dataflow. We noticed that there is a way to interact with the API for our Domo instance by using first an authentication method. However, when we tried to use that method (either in Python scripting or in Postman) it returns…
-
Reverting Back to the old dataSet version
Hi team, I changed a few things in my recursive ETL dataset and lost all the rows of data that were recursed in it. When I try to revert the flow to a previous version, it does not show the lost data rows. Any way to retrieve those?
-
How to remove empty columns from a dataset?
I am using a DOMO connector to import Salesforce data. The imported data has 30+ columns with no value. Can a query be written to remove all the columns with no values? I know I can write a script using Python/R but we don't have a scripting tile in our package.
-
Using a View as a input dataset to mysql dataflow
Hi All, I am trying to have (a view) as one of the inputs to the mysql dataflow...But its not being accepted ...Can you please help me with this... Thanks, Arun
-
Domo CLI - Return Limit for List-Dataflow Command?
when running list-dataflow --filename filename.json, I believe that the returned payload only contains 100 dataflows as a hard limit. There aren't args to pass that allow me to offset the object index. Has anybody encountered this problem and/or have a way to get all dataflows from Domo CLI
-
Creating a join from a lookup table
I have created a dataflow that joins two sets of data together, however when i join them together it's dropping data because it doesn't have anything to match with. How would i create a dataflow that would match all the data and any of the unmatched records would also stay?
-
Moving one column under another
I have a table with 5 columns: column 1 is an ID, columns 2 and 4 are codes, and 3 and 5 are corresponding names. I need all the codes in one column, and all the names in one column. These means each ID will now have 2 rows, one for each code/name. How would I go about moving data from one column to another in Magic ETL?
-
Drill Path Filter to show Specific Apt's without a Specific Procedure
Hey guys, I'm worried that the answer to my question is that it has to be done in ETL, but I'm hoping it can be done as a beast mode. So the data in question looks similar to what I have below: Doctor | Procedure | Date | Apt # | Patient Dr. B | D0330 | 4/1/22 | 80136 | I. M. Madeup Dr. B | D0126 | 4/1/22 | 80136 | I. M.…
-
DomoR vs RdomoSDK
I have been handed an r script that has is using an accessToken to connect to domo and extract a dataset using a dataset ID. My end goal is to translate this to Python, as I have an existing pipeline in python that needs this dataset. Initially using the naive approach I tried to connect to Domo using pyDomo but was…
-
Rolling Average for Each Category
I have a dataset containing total monthly units by month and by person. I'd like to calculate the 3 month rolling average for each person. How should I group the data so that the rolling average calculation goes by person and not by date?
-
Best Practice: ETLs Vs SQL ETLs
We are trying to rebuild a tool Domo Tech built for us called Domo Stacker. Its main purpose is to aggregate many datasets together. Our current datasets are 250+ recursive ETLs that output a dataset that is growing (think: historical + appended new data) for four buckets of categorized data (four separate outputs when…
-
Adding business days to a date via MagicETL
Hi, I'm currently calculating a deadline using a dynamic formula in MagicETL. It looks like this: CASE WHEN `issue` = 'bug' THEN ( CASE WHEN `priority` = 1 THEN ADDDATE(`CreationDate`, INTERVAL 4 DAY) END ) The calculation works great, but I was wondering what would be the best way to update it to 4 business days, rather…
-
Creating complex ranking using certain values within a dimension
Hello all! I am attempting to create a sankey diagram that shows a customers activities leading up to a certain conversion. Each customer can have multiple conversions so I want to rank each activity leading up to a conversion, and have that rank reset after the conversion happens. Having some trouble theorizing how to do…
-
Right string operation to remove the timestamp
I have a long date that I have converted to a text. Now I want to use the string operations tile to cut off the tail so I can have a nice clean date. the way it looks now in a string format is: 2022-02-24T12:08:34 What I want is: 2022-02-24 What I have been trying is: But it leaving the date unchanged and that timestamp at…
-
Most efficient way to export data from AWS to Domo with little to no effort?
Currently using Redshift/QuickSight for data analysis. Would like to export data to Domo with minimal effort. What are the pros and cons for each option? Thank you.
-
Framed Sum in Rank & Window Magic ETL not treating zeroes properly
Hi there, I'm trying to produce a rolling 12 month total for each month in a dataset, but I'm finding with some of the partitions that I get a negative exponential when I'm expecting it to return zero (all the months are zero - see raw data screenshot). It doesn't seem to happen with every partition that has zero values,…
-
More easily jump to a Dataset's Dataflow
When in a Dataset I want to easily jump to the dataflow that created the Dataset. It's sort of hidden over here. Why not move the "Edit Etl" down to this section?
-
Trying to condense / group rows to remove nulls, any help appreciated
Hello, I am working on a data flow that needs to perform various substring functions to isolate data from a large xml string. I have gotten it close, where the data is linked to the log / time / user in in the sql database, and i am able to separate values into appropriately named columns. I would like to know if/how to…
-
Data Overview
Dear Community. I am trying to get an overview what data we are currently importing to DOMO due to data privacy evaluation. There are a lot of Data Sources (54) and a huge amount of respective Data Flows (1.9K). My question is if there is a functionality or app supporting me with listing all data schemas/column names per…