-
Alerts on Output Dataset Fail with False Claim of Duplication
When attempting to add an alert to the output dataset of one of our dataflows we receive the below error message: We have added a Remove Duplicates step to the dataflow on the `Case Number` column just before the output dataset, however this message is still being received. We even compared the Count to the Distinct Count…
-
Reconnecting Disconnected Output DataSets and Cards
We are developing under several accounts (e.g. Dev1 and Dev2). In order to organize the ownership, I have changed the ownership of some pages and cards to one account (Dev2 > Dev1). Then the connection between the DataSet and the card got disconnected and the card is no longer visible. For the card with a direct connection…
-
How to append data to the output dataset?
I have a workbench job that will run daily. It will get the data with current date. I then needs to filter this data and create two data sets using the DataFlow ETL tool. How do I append data to the DataFlow ETL output table? DataFlow ETL creates a new table every time it runs. I realize that One way to achieve this is to…
-
Reg. Expression to replace a list of names with a single character.
I have a column that is in a string format, and of course contains names. Majority of the rows are an empty string. What I am looking for is to take the rows that do contain a name and replace them with a 1. i have been trying to use replace text tile then trying to use a regular expression in the 2nd step, then replace…
-
Changing a value to an integer in the ETL
I have a column that uses an 'x' as a flag, any row that isn't flagged with an 'x' is left as an empty string. What I need is to replace the 'x' with a '1' to be used in a later count. this should be straightforward but I keep getting my wires crossed with the order of operations.
-
Where can I find the logic for custom built dimensions?
Hello, I'm having an issue tracking down where, exactly, I can find the logic for these custom-built Dimension Segments. Any tips for where I might find it? Thanks!
-
Redshift Unable to create database.
We are getting the below error when trying to build a Redshift dataflow: We are unable to run previews, and we are unable to run the dataflow itself. It fails every time we try to run. Any idea what is causing this issue?
-
what does the delta symbol mean on my data flow ?
I checked all the flows and tables involved in building this flow, but I cant find any mistake or warning. What does that orange delta symbol mean ? it doesnt give any info when i hover over it or when i click it. thanks for the help.
-
Sharing favorite filters in data center to colleagues
Hi all, I have been using tags and favorite filters as a way to group my team's datasets and dataflows into collection. Just until today I realized that the favorite filters I created for my dataflows are not visible to my colleague (while, strangely, the filters for datasets are). My colleague and I are both in the same…
-
"Invalid Schema, Check Your Dataset Query" error
Hello, I only saw a few questions related to this on Dojo and figure I might ask. My dataflow has always ran fine (created in May 2021) but yesterday there was an error that showed up. I'm not able to preview the dataset, and I tried to refresh the ETL multiple times and reset the schema (deleted all columns from the…
-
Appending an update set to itself
Hi, I have a MySQL dataflow that is built from API reports from a vendors application. The idea is that we did a one time historical pull of the API as dataset 1. Then, the same API that only pulls 1 day of data and appends to the historical dataset as dataset 2. But, every once and a while it seems as if data is dropping…
-
Notification of DataFlow and DataSet Processing Failures
I'd like Domo to notify our admin team's Google Chat or mailing list when a DataFlow and DataSet fails to process. I know that it can be configured in "Notifications" from the top right corner of each DataSet, but I don't think it is possible to notify any email address other than the owner. I have also not found a webhook…
-
Row Numbers from original data
Hi there! Huge fan of the forums and hoping you can help me with something that previous posts and many late night Google searches have eluded me on... We have an email connector pulling in many CSV files from attachments and appending them one after the other in the order received. The order of the data within each CSV is…
-
Long Execution
Hi people! I'm here again! This time, i have a e long execution problem! I have a table with raw data with 46 million lines, approximately. Here is my first question: When a create a dataflow that process that 46 million rows and generates a output dataset, that dataset is a physical table or anytime i use that output…
-
Dataflow creation best practice and recommendation
A lot of the dataflow creation I do is taking 2 datasets that differ slightly and don't have an easy 1:1 comparison that makes joining the data very easy. usually I am using 1 report for almost the entire visualization, and puling a column or two form the 2nd report. Since the reports aren't in an identical format, and…
-
Join Function in ETL
Hi, I am trying to join a table on my DataSet, using the left join function. My original DataSet contains only 118k rows, and the other table contains 1405 rows. I thought that the output DataSet will stay 118k rows, and just have addition columns on it. However, the output DataSet is 4.6M which is way more then I…
-
How to split values from one column in to 2 columns?
I have a column that contains values that are 4 digits. I want to divide them up in to 2 separate columns. The first 2 number are the % of the discount, and the last 2 are the length Ideally it would be the % and contain 30, 40, 35, 40 ,etc... Then Length would have the last for digits like 02, 02, 03, 02, 04, etc.. I am…
-
How to delete portion of text in a column in the ETL?
I have a date column that is in this format: The format of the date is fine, but what I am trying to do is get rid of everything including the 'T' and to the right, so that all that is left would be 2021-06-27. If a solution within the ETL is possible that would be great.
-
Data Flow help
Hello, Experts, I am humbly reaching out for some help. I have a vendor sending a snapshot of a database table every day and not sending me diffs. I am trying to determine the diffs. The connector type is SFTP CSV. The file they send daily is overwritten daily. I currently have the data connector appending the data from…
-
Share dataflow and dataset owned by me to other author in the same organisation
Is there any easy way to share everything so that the other authors can see my work? I need to click share each time for each dataflow or dataset or cards that I created. How can I work more collaboratively with other author in our instance so we know what's the progress from each other? Thanks Sherry
-
New addition to Webform not flowing through all dataflows
I have a webform that houses in different columns site level details like store name, Region, and Site ID's across multiple sources. I use this to join with datasets so it makes it easy to filter for things like Region and Site's. I recently added a new site, and all the other details to this webform. if has flowed through…
-
Exclude first Value - Rank & Window
I am currently calculating the contribution for my dataset and have already sorted my values in descending order using the rank tile with date and group for my partitions. What I'm hoping to do is sum the previous value + the following value until my last value would be the grand total. In theory, I thought using the rank…
-
Deleteing intermediary Dataset values after final OutputDataset
Hi I Am new to Domo we have a lot of very large etls that are broken down in stages. after each stage an intermediary dataset it created to the pont we have billions of rows in datasets that are only used as staging tables. my question is these tables take up space. once the final table is created, how can we delete all…
-
Does ETL Support the 'IN' Function in a case when?
Does ETL Support the 'IN' Function in a case when? Does anybody know if this is possible? I'm getting an error I just want to make sure that it's not my syntax.
-
Convert DATETIME (UTC) to specific timezone
Hello, Domosapiens! I have an interesting quandry. I have a data flow (using MagicETL - should I be using something else?) that I am trying to build and convert a date that my MySQL data connector is bringing over. The datetime column comes from an orders table and is called posted_date and it brings the data over as UTC…
-
Aggregating Rows
I have a dataset that breaks everything everything out by rows. any time there is a rental, or vacate it adds a value in to a new row with a siteID for a store. the data refreshes on a daily basis, so it site 3196 has 4 rentals in a day it would repeat site 3196 on 4 different rows with a value of 1 in the rental column…
-
Improve Dataflow Efficiency
Hello, Domosapiens, I currently have a data flow that is taking 14 min or greater to run. The design is as such: MySQL connector that first runs once to pull in all historical data (7.8M rows). Ex full historical pull query: select * from table; change dataset settings to merge method by changing the query to select * from…
-
What is wrong with my Pivot - Uncollapse Columns? Showing Null
Hey I'm trying to pivot a column, but the new columns are all null Here is the original data This is how I've set up the pivot info This is the output I get: Originally not every Customer_ID had ever Level(1/2/3) I thought this was the cause of the nulls but I've fixed it so every Customer_ID has all levels and still the…