Data Flows

Magic ETL Won't Save DataFlow?
Team, I have built an extensive ETL and I want to "Save and Run" to explore and analyze the output. Unfortunately, the "Save" button isn't responsive. I will click on it and it won't actually save. I'll click "Save and Run" and it won't actually save… Weird issue that I've never ran into before so hoping there's a simple…
Change Certification from Datasets to Dataflows
Certification is currently requested and managed via a dataset, however we've found that in some cases changes to the dataflow don't cause the certification to expire. This introduces potential risk in cases where changes could be made that affect the accuracy of the data, but the dataset could still be flagged as…
Add time zone option to inbuilt date/time functions
The Domo functions such as NOW(), CURRENT_TIME(), CURRENT_TIMESTAMP(), CURTIME() all return times in UTC. It would be great if these could all be modified to take additional optional parameter of time zone so that the return values are shifted to the local time appropriate to the given time zone (including daylight savings…
Overage Execution Rows Protections
Just had a user accidentally do a join that we identified 7 hrs after the process started that was up to 72 billion rows. It was going to top out over 300 billion, but we hit cancel. However, that overflow credit usage topped out over 3,000 credits, which is not the kind of mistake you'd like your team to be making. So,…
Group By tile not working with large dataset
Hi, I have a dataset of about 4.7M rows. When I try to use the Group By tile in Magic ETL to find a sum of values across IDs, and then join back on the ID, the new sum column is blank. However, in testing, when I filtered the dataset down to about 50K rows to see what was wrong, the example ID I was checking now had the…
PayCom Connector?
Hi Domos, Does anyone have experience connecting PayCom (HR & Payroll software) to Domo? There is no canned Connector in the AppStore.
Data Join Confusion - Magic ETL
Hi, I'm trying to join two sets of data together (assignments submission data & assignment summary data) For the output, I am trying to get a list of all assignments with a due date in a date range that have not been submitted by the students. The assignment submission data has a row for every student that lists every…
Display Record Count in Magic ETL on each step
Tl;dr - display the record count in MagicETL on each step of the last successful run of the ETL. Detail: I often spend quite some time diagnosing and fixing ETLs. In larger ETLs, it can be very onerous to determine whether a record with an expected behaviour makes it through the entire flow to the end to validate the ETL.…
Provider Prior to Upgrade
I am trying to determine who provided the evaluation prior to an upgraded premium procedure. I have all the data I could ever need, but I'm not sure of the logic to make it happen. The task is made more difficult because there could be multiple evaluations prior to a procedure, and evaluations on different dates than the…
DOMO SQL - Formula
Hi Team, The formula below is validating but it is showing "NaN" for all CPM values. For context estimated_youtube_ad revenue and ad_impressions may include the following values: 0, Null values, integers greater than 0. Any potential way to fix this? Thank you in advance! Select* ,…
DataFlow Error: Trying to Schedule DataFlow in Jupyter Workspace
Has anyone successfully resolved this error when trying to schedule as a DataFlow in Jupyter workspace? I don't have any other DataFlows scheduled in this workspace.
Scheduling Dataflows with Multiple Notebooks Within a Single Jupyter Workspace
You can only schedule one dataflow per workspace. Documentation is misleading saying you can schedule per notebook. Not true. https://domo-support.domo.com/minasan/s/article/36004740075?language=en_US#schedule_notebook The moment you open a second notebook and try to schedule it from within the same Jupyter workspace you…
Export Magic ETL Code for External Version Control
In our current environment, we use an external repository (Bitbucket) to maintain version control and the ability to run detailed diffs before making changes to our SQL dataflows. It would be helpful to have a way to export either the code or some representation of the current state of a Magic ETL dataflow to be used in…
Removing Dataset Outputs from Magic ETL
We have been cleaning up our Domo instance removing unnecessary outputs from Magic ETL flows. I am assuming these datasets aren't truly being deleted, just disconnected from their source to not get updated any more. Is there a way to find these "orphaned" outputs so that they can be properly deleted?
Creating a Beastmode/ Dimension
I have a set of data that includes columns like " Facebook Engagements, Facebook Impressions, Twitter Engagements, Twitter Post Impressions, TikTok engagements, TikTok Impressions" how do I create a dimension or beastmode where you will be able to filter by platform, e.g.: Facebook, Twitter, TikTok.
Does domo allow a dataflow to run for more than 24 hours
I have dataflow that gets failed when it tries to completely run a dataflow that runs more than a day. Is there any solution. can the dataflow can run more than a day. Can anyone help me with this problem?
How can I mark a duplicate row instead of deleting it in magic ETL?
I use domo to handle metadata transformations for music tracks, and for publishing purposes it's important that I don't have any duplicate track titles. It can mess with the royalty payouts. So basically I want to mark a row as a duplicate based on specific values in columns/rows instead of deleting it. Ideally I'd love to…
Add ORDER BY to GROUP_CONCAT Function in Magic ETL v2
Currently, Magic ETL v2 supports the aggregate function GROUP_CONCAT in its Group By tile. While clauses inside the GROUP_CONCAT function like DISTINCT and SEPARATOR are available, the ORDER BY clause doesn't work and causes the tile to fail. SQL DataFlows currently support using the GROUP_CONCAT function with the ORDER BY…
Find Datasets with No Cards & that are Not Used as Inputs (dataflows, dataset views, fusion, etc)
I'm working on a card to identify unused datasets. Datasets that meet the following criteria: No cards powered Not used as input in any other dataflow (ETL, Fusion, Dataset Views, etc/all types) I did find "Datasets" & "Dataflow Details" (Domo Governance Datasets Connector) that have most of what I want but does not…
Mega-Ultron-Super-Duper Connected Dataset to rule them all
BLUF: Should I join/append all of our data so we have one workable dataset? This isn't a technical question, but rather theoretical. We've got data in many places (I'm sure this is common): Budget in XYZ Financials in ABC Stats in DEF Stats Historical (just switch systems so have a different datamart) in LMN Everything…
Data storage
Hello, I understand any data coming from files and output datasets from DOMO ETLs shall store data in DOMO Cloud. Any of the following shall also store in DOMO Cloud? DataFlows SQL View (DOMO) Adrenaline Transform What if I use federated connection as my connector? or an API to retrieve data from source and leverage any of…
Is there a way to track which DOMO assets (cards, datasets, dataflow, etc) uses a specific field?
We found the Card Fields & Beast Modes Governance dataset which shows what fields are being used in cards and beast modes but we would like to track a field name from it's original dataset > name change (if it applies) > DOMO asset.
Where is my data going??
My data seems to be disappearing between the ETL and the final data that it actually outputs. Here are some pictures to try to explain what's happening. The first image shows the preview on the final tile in the workflow (being the output file titled Work Numbers (Final)). The second image shows the same chunk of data,…
Notification when dataset or dataflow runs
Hello, Have some clients asking about an alert when a dataset or dataflow has completed. Currently we just have failed notification but clients would like to get a confirmation on vital dataflows/datasets completed, during immediate need moments. Helps when we are using the DOMO cloud to pull data from DOMO into other…
Budget to Actual - Multiple Dimensions & Different Datasets
I have a budget dataset with the following columns: _Account _Subsidiaries _Department _Doctor _Location (Not in Use) _Headcount Resources _Placeholder 4 _Year _Period _Scenario _Currency _Measure _value I have an actuals dataset with the following columns: Appointment Date Status Appointment Type Patient Account Number…
Open dataflow version comments for edit after the save
We have a difficult time getting people to document the changes they have made to a dataflow. I would like the ability to go back and document a certain version of a dataflow after it has been saved.
Pivot/Group By - Make Multiple rows into one row and add columns
Hi Community - thanks as always for the consistent great help. I'm attempting to Group/Pivot/Join/Append (not sure exactly what the methodology will be) multiple rows of Patients into single rows with additional columns. Right now, each patient event is on it's own row. So, a patient seen on Monday and then also on Tuesday…
STR_TO_DATE throwing Parsing error in ETL
I am facing a error while converting string to date. This process is done in the ETL Date column has value - Jun 11, 2024 @ 14:33:22.000 Formula - STR_TO_DATE(@timestamp, '%M %e, %Y @ %H:%i:%S.000') The error showing is as attached below Note : We receive data using email connector. Please let me know where it is going…

Data Flows

Discussion List