-
Dataflow creation best practice and recommendation
A lot of the dataflow creation I do is taking 2 datasets that differ slightly and don't have an easy 1:1 comparison that makes joining the data very easy. usually I am using 1 report for almost the entire visualization, and puling a column or two form the 2nd report. Since the reports aren't in an identical format, and…
-
Join Function in ETL
Hi, I am trying to join a table on my DataSet, using the left join function. My original DataSet contains only 118k rows, and the other table contains 1405 rows. I thought that the output DataSet will stay 118k rows, and just have addition columns on it. However, the output DataSet is 4.6M which is way more then I…
-
How to split values from one column in to 2 columns?
I have a column that contains values that are 4 digits. I want to divide them up in to 2 separate columns. The first 2 number are the % of the discount, and the last 2 are the length Ideally it would be the % and contain 30, 40, 35, 40 ,etc... Then Length would have the last for digits like 02, 02, 03, 02, 04, etc.. I am…
-
How to delete portion of text in a column in the ETL?
I have a date column that is in this format: The format of the date is fine, but what I am trying to do is get rid of everything including the 'T' and to the right, so that all that is left would be 2021-06-27. If a solution within the ETL is possible that would be great.
-
Data Flow help
Hello, Experts, I am humbly reaching out for some help. I have a vendor sending a snapshot of a database table every day and not sending me diffs. I am trying to determine the diffs. The connector type is SFTP CSV. The file they send daily is overwritten daily. I currently have the data connector appending the data from…
-
Share dataflow and dataset owned by me to other author in the same organisation
Is there any easy way to share everything so that the other authors can see my work? I need to click share each time for each dataflow or dataset or cards that I created. How can I work more collaboratively with other author in our instance so we know what's the progress from each other? Thanks Sherry
-
New addition to Webform not flowing through all dataflows
I have a webform that houses in different columns site level details like store name, Region, and Site ID's across multiple sources. I use this to join with datasets so it makes it easy to filter for things like Region and Site's. I recently added a new site, and all the other details to this webform. if has flowed through…
-
Exclude first Value - Rank & Window
I am currently calculating the contribution for my dataset and have already sorted my values in descending order using the rank tile with date and group for my partitions. What I'm hoping to do is sum the previous value + the following value until my last value would be the grand total. In theory, I thought using the rank…
-
Deleteing intermediary Dataset values after final OutputDataset
Hi I Am new to Domo we have a lot of very large etls that are broken down in stages. after each stage an intermediary dataset it created to the pont we have billions of rows in datasets that are only used as staging tables. my question is these tables take up space. once the final table is created, how can we delete all…
-
Does ETL Support the 'IN' Function in a case when?
Does ETL Support the 'IN' Function in a case when? Does anybody know if this is possible? I'm getting an error I just want to make sure that it's not my syntax.
-
Convert DATETIME (UTC) to specific timezone
Hello, Domosapiens! I have an interesting quandry. I have a data flow (using MagicETL - should I be using something else?) that I am trying to build and convert a date that my MySQL data connector is bringing over. The datetime column comes from an orders table and is called posted_date and it brings the data over as UTC…
-
Aggregating Rows
I have a dataset that breaks everything everything out by rows. any time there is a rental, or vacate it adds a value in to a new row with a siteID for a store. the data refreshes on a daily basis, so it site 3196 has 4 rentals in a day it would repeat site 3196 on 4 different rows with a value of 1 in the rental column…
-
Improve Dataflow Efficiency
Hello, Domosapiens, I currently have a data flow that is taking 14 min or greater to run. The design is as such: MySQL connector that first runs once to pull in all historical data (7.8M rows). Ex full historical pull query: select * from table; change dataset settings to merge method by changing the query to select * from…
-
What is wrong with my Pivot - Uncollapse Columns? Showing Null
Hey I'm trying to pivot a column, but the new columns are all null Here is the original data This is how I've set up the pivot info This is the output I get: Originally not every Customer_ID had ever Level(1/2/3) I thought this was the cause of the nulls but I've fixed it so every Customer_ID has all levels and still the…
-
Is it possible to use while/do loops in MySQL in Domo Dataflows?
I have data that has varying rows of data for each identifier, from one row to 26. For each identifier, I have four different columns that change with the various rows. For each new identifier, I want to loop through the rows that follow that have the same identifier and build the row out with the unique values in these…
-
How can we use similar function as Vlookup??
I would like to offer a volume discount price for each company A, B, C and calculate total biil. There are 2 datasets like the below. When we sell 250 of Products to Company A in 2021 Feb (Dataset B), I would like to apply Price 4 USD on the price list for Company A (Dataset A). Then the total bill will be 1000 USD. If…
-
ETL
I have a column with multiple categories values, I want to convert this categories values to column labels. How do I do this. In the screenshot attached below I want to convert values of column '123 1' to column labels.
-
How to implement commands from the Command Line Interface Tool in Python, specifically api requests
Question: Is it possible to execute command line interface tool operations within Python, specifically the 'list-dataflow' command? Elaboration: The 'list-dataflow' command in the command line interface tool returns JSON structured metadata for a dataflow. My guess is that this command is just making a GET request to a…
-
DataFlow Transform (i.e. Action) Types in API JSON data
My question: What factors determine the type of a dataflow transform in the api JSON data (e.g. GenerateTableAction, SQL, SqlAction)? Background: I am using DOMO's command line tool to get JSON structured metadata about a DataFlow. Specifically, I am using the list-dataflow command: Putting in the DataFlow id and a…
-
Timestamp Convention in JSON File from Command Line DataFlow Request
My question: Does anyone know DOMO's timestamp convention and how these map to normal timing conventions? Background: I am using DOMO's command line tool to get JSON structured metadata about a DataFlow. Specifically, I am using the list-dataflow command: Putting in the DataFlow id and a filepath returns a json file to…
-
"Failed to start data processing engine"
Hi, I get this error message across various datasets a few times a day. It only occurs from scheduled/dataset updates. It's not a major problem as the dataflow succeeds on the automatic retry, but the notifications can be annoying. Has anyone else had this issue? Is there a fix for it (or can I at least remove the…
-
How do i trim out an email address from a field with a string?
How do i trim out an email address from a field with a string? Example of Field: Yes, we accept this current pricing and would like to proceed with this order. Thank you, Bob Dole Lab Research Analyst Poss Lab—Dole University bob.dole@some.edu Thanks
-
Is it possible to do CTE hierarchical recursive query in MySQL?
I have a table with thousands of rows in which there is a relationship of child/parent between the rows. For each record I would like to have all the hierarchy related with that record. As an example, I have the following table and for the id=19 I would like to return all its related children, which are [20, 21, 22] I have…
-
MySQL Data Flows clarification: MySQL version & window functions (row_number)
I've neglected MySQL Data Flows because I'd passingly assumed that they were for MySQL-derived DataSets. It seems not, it looks like a MySQL DataFlow is a DataFlow that uses an embedded version of MySQL. Is this correct? I've seen mention of Window Functions in passing, and assumed the MySQL must be 8.0+, as that's when…
-
Python
Hey guys, I can't find any documentation on how to use the python widget. I've read the documentation on the site already. It didn't provide me any insight on actually implementing it and the Schema tab is far too confusing. I don't even know why I would import all my selected columns if I'm going to be creating another…
-
Empty cells after aggregating and creating a case when variable
Hi I have a dataset that shows a sale/lost sale an agent made on a daily basis. Now I am using the dataflow to get the sales % over the last 3 months (data is only last 3 months) per consultant and want to join it back to the original dataset for later card creation purposes. This following formula gives me empty values…
-
Create Task on Dataflow Output by Row
Hi! If this isn't the right forum to post this, please let me know. TL;DR: I'm trying to automate alerts/tasks from the output of a dataflow such that each row in the output gets its own task instead of the entire updated dataset getting one bulk "dataset is updated, here you go" task. Two failed attempts are detailed…