-
Make MagicETL Dataflow Formula Preview Match Data Output
in some cases when running the preview I get one result and when I run the dataset I get another. For example I have a formula SHIPDATETIME defined as " ADDTIME(DATE(`SHIPDATE`), TIME(`SHIPTIME`)) ". When I run this in the Dataflow preview I get the expected result when I run the dataflow the output dataset does not match…
-
Data Flows - Testing Use Case Filters
Use Case: When building data flows with data sets in millions of rows, it's hard to test if the logic works across those rows unless I add filter tiles that filter out data at specific points during the Data Flow build process. This can add complexity when adding many test cases at specific points. The Idea: It would be…
-
Magic ETL: Avoid loading all the input columns and rows from data vault.
In Magic ETL, to avoid loading all the input columns and rows from the data vault. On the Input tile, under “Data Handling Section” if there could be a provision (refer to attached pdf with screenshot) check box to select required columns (simple request) and text box to add a where clause on each column Or one text box to…
-
Ability to remove unnecessary columns directly in the input DataSet before loading all its rows
The functionality I require is already available in the MySQL and Redshift DataFlow… …in this example deselecting "department" and "state" columns. This option would make it possible not to load all columns for all rows in the DataSet. I hope it will be available in the Magic ETL input DataSets as soon as possible. Thank…
-
DataFlow Notes - Output Datasets
I would like the DataFlow Notes on Output Datasets to be available in the DataFlow's /datasets view (possibly using the same yellow note embellishment icon as used within the DataFlow.)
-
Does domostats or governance connector have data points on data flow's input/output storage amount?
Hello - Does domostats or governance connector have data points on data flow's input/output storage amount? For example, xxx MB/KB/GB I highlighted in the screenshot below?
-
Adding new Historical Data to an Already Existing Recursive Dataflow
I am looking for a way to add historical data into my already existing recursive dataflow. My two input datasets are 1.) an email connector that uploads once a day, 2.) a copy of my output dataset as an input dataset acting as my historical records. My output dataset has data from 2021-2023 but I would like to add data…
-
Include Data for a specific date range
Hello - newbie here - I need to know how to only include data that falls within my defined start and end dates columns. There are multiple flight dates, so I need to be able to bucket the daily delivery data within those defined dates. It just pulls in all of the data for each flight and I cannot seem to get it to work. I…
-
Removing duplicates
Hi, I need help coming up with logic for this problem. I have a dataset that looks like this: ID Source 1 A 1 A 1 B 1 B 2 A 3 B I want to only remove rows from source A IF that same row is also in source B. Otherwise I want to keep rows unique to source A and unique to source B. The output should look like this: ID Source…
-
Approach to logically group data via inference of similar text strings
Hey all. I'm not looking for the detailed solution here as I know it will be complex, potentially require purchased add-ons and most definitely will require my dev team. But I'm hoping someone can guide me to general approaches and Domo capabilities that would be leveraged - if even supportable. We've got a large data set…
-
Is there an alternative for the add constants tile?
Hi Everyone, I'm trying to establish a new column with the same value for every row. But instead of a manual floating decimal input through an add constants tile, I'd like to generate the value from the magic etl auto calculation every time the dataflow runs. I know there is a simple way to do this, but the solution…
-
How to Unpivot or Explode Variable-Length Lists in Domo to Flatten a Table?
Hi Domo community, I have a dataset in Domo where the number of postcodes in a cell can vary. Here's an example of my input data: Network State Postcode A VIC 3000, 3001, 3002 B NSW 3011, 3021, 3001, 3004, 3012 I need to transform it into the following format: Network State Postcode A VIC 3000 A VIC 3001 A VIC 3002 B NSW…
-
Comparing Datasets to Determine Variance
Hello everyone, I am working on comparing data from a previous data input, Universal Analytics, to a new one, Google Analytics 4. Basically, there was an update and data is formatted differently. I am attempting to join the two datasets to compare the month of A ugust for variances, but the closest thing to a unique column…
-
Long Running Dataflow Alerting/Auto-Restart
We need to have the ability to understand when dataflows are running longer than the average duration. We would like to set user defined alert criteria based upon the dataflow's duration. For example: alert when dataflow has been running 15min, 1hr, etc. longer than the average for the dataflow. Having the functionality of…
-
Dataflow failed
Hello all, I get the below error after running my dataflow. As this consists for multiple inputs, I haven't still been able to trace the exact field causing it. But I do have a formula tile with this calculation.. " Failed to convert value '-3--3-01' from type 'String' to type 'Date'. " Some days this data flow runs…
-
Dataflow Search Screen - Show # of Tiles
It would be cool if when searching for dataflows that screen could show # of tiles in the dataflow.
-
Data transformation ideas around old/new user data
HI, we're migrating from username sign in, to okta that uses email as user id. we have an existing user info data with user id as username (xfsgsbss) but we would have a new data with user id as email(fdhsvbsjsbsjsh@email.com). I want to be able to join both old and new data and make sure the right attributes goes to them,…
-
How to create a rate by specific selection criteria in Domo
Hello again community. So I work within a healthcare organization and I have a domo grouped/stacked bar graph built currently from an etl that has columns regarding patients readmission start date, account number, discharge date and attending providers(previous admission). The card breaks down the readmit percentage rate…
-
More info in logs
It would be very useful if, when executing a Domo ETL, a log file was generated to see the details of potential failures in the process. Currently, if there is a failure in any of the Domo's own boxes, you see that the process has been interrupted in a part of the process but we don't see the details. Also, if the process…