-
How to remove whitespace after a full name in ETL
I have a name column with a full name: example Bob Dylan and am joining to another dataset by name. I'm having issues and believe its due to extra white space after the last name. I tried using Replace text, regEx, \s, but that just removed all white space creating BobDylan. Any thoughts here?
-
Can I get the MIN date over a partition with a formula tile?
I need to walk a date column partitioned by the value of another column (UID) to find the earliest date for that group. I do not want to collapse the rows. I've tried a number of approaches to this with a SQL perspective, but I'm starting to think Magic ETL wants me to use a Group by. Here are a couple of examples of my…
-
Preview flow "from here" option?
I think I know the answer, but I thought I'd check anyway … Is there any hack to cause a Magic ETL preview to run from a certain point in the flow forward? I'm working on a flow that starts with some heavy sources and groups down before performing some merges and other heavy actions. I would LOVE to preview from a point…
-
Add jump points to ETLs to clean up cluttered ETLs
When I create an ETL I do my best to make sure that all logical paths are kept together and that paths from one tile to another don't cross other paths if possible. Sometimes, however, I can't help but end up with a rat's nest of paths that clutter up the ETL and make it hard to follow. I would like to request jump points…
-
Magic ETL History Error Log Visual Representation
It would be great if in the history of the dataflow it gave some sort of visual representation on which FAILED had an actual error message I normally have to look at each one to find the one with the actual error
-
Average of a difference in Timestamps
I have two timestamp fields: completed at and delivered at. I want to find the difference of both and take the average. The dates include a window that is sometimes over a day. I read this By @GrantSmith which is useful. Is there an easy way to do this without breaking up each segment (day, hour, minute, second) then…
-
Changing dataset update method from Replace to Partition doubles data
We've noticed that if we change the update method of an existing dataset to Partition from replace, we end up with two records in the dataset for every new one (and both of them look identical, down to the field we're using for partitioning): it was quite the shock to see a dataset I expected to have 91 million rows…
-
Issue with Sales Data Tracking and Date Transformation
Hello, I'm encountering an issue with tracking sales data in from our CRM. The problem arises from the fact that the table card relies on the 'Date Created' column for capturing engagements. Although this successfully captures most sales data, it fails to include engagements that were created in the previous year but…
-
Formulae function for table instead of line by line
I have two tables. The first table i have a list of telephone numbers. The second table, a list of the first 3 digits of the number. I want to qualify the first with the next. Here is my formulae: case when SUBSTR(Phone,4,3) not in(NXX) then 'no' else 'yes' END so this is fine in that it qualifies line by line. However, I…
-
Why doesn't the Date Settings feature in Magic ETL update my dates?
When I navigate to Settings and change the Timezone to America/Los Angeles, it doesn't change any of the values for my date columns. However, when I navigate to my input dataset in the ETL and change the time zone on an individual date field, it works as expected. Am I doing something wrong?
-
Fill in Blanks based on existing column
Hi All, Financial Analyst trying to tackle ETLs. Vaguely familiar and can work my way through MagicETL but am stumped on this one. Hoping for some help. I have a data set replicated below. I want to fill the the [BLANK] using an ETL so on the data set where the store numbers are the same and the ticketIDs are the same, the…
-
Chose the newest date available
Hi All, I am working to combine two tables. The first step I need to take is to filter the project_number by with the newest report date available. The example below shows two different project numbers The first one has the last report date as Q3 2022 - we need to capture that input The second project_number has q4 of 2022…
-
unpivoting data
I have data this way: Account | Category | Date 12 | Phone | 01/01/22 12 | TV | 01/01/22 13 | Phone | 01/02/22 12 | TV | 01/01/22 I would like to figure out on each date the counts for each subcategory. First i would want the data pivoted this way: Account | Category | Date 12 | Phone, TV | 01/01/22 13 | Phone | 01/02/22…
-
Does anyone know a Regex that only accepts a segment with a number before 'k' and nothing after?
Does anyone know how to create a Regex that only accepts a segment with a number before a 'k' and nothing after? Or it can be a regex that replaces everything that doesn't have a number before 'k' and nothing after with an empty string. I am looking to extract the circ value for our email campaigns but some rows have a…
-
Pass Variables into a MagicETL from Workflows.
With the new workflow features on the horizon, I think it would be immeasurably valuable to pass output variables from workflows into Magic ETLs since workflows can execute ETLs already. For example: An 'audit' workflow is triggered that evaluates a dataset for various criteria. As part of the evaluation it sets output…
-
Word Count
Is there a way to create a field or value that identifies the number of words in a field? Preferably ETL, but Beastmode if necessary.
-
Sum of units sold, rolling 30 days
Looking to replicate the exact data layout above (side by side columns) in a table in Domo. So units sold by SKU by month (got this far) but also a column for rolling L30 Days sum. I created a new column in the dataset for count of days from current for each row DATEDIFF(CURDATE(),(Calendar Date)) but having a challenge…
-
Averaging time/returning only whole numbers of a fraction number
Is there a way in a beastmode to make the following example numbers on the left below remove the numbers after the decimal? 14.1925 — 14 15.999 —- 15 0.98 — 0 I am trying to get the average time of day that donors answer their phone when we call. So here below this donor answered his phone at these 3 times (hh:mm:ss):…
-
Using partitioning in ETL or Workbench
Good afternoon. Typically, to not do full replaces of data pulls into Domo from our databases, we have used recursive dataflows with ETL. I am trying to think of a way of accomplishing this using the partitioning features in ETL and/or workbench to avoid the ETL still having to process the whole set of updated+historical.…
-
SQL formula not working?
I have a series of tests that run to determine a pass or fail. All the formulas work but this one is giving me problems - CASE WHEN Critical/Major= 'Common' AND
No>= 2 AND Critical/Major= 'Major' AND No>= 1 THEN 'Fail' Any one know why this would not be returning a Fail?
-
Average Time between Orders
Stuck doing this in Magic ETL: Trying to calculate the average time between orders. For each record in the DataSet, add a field showing the number of days since the previous order for that customer. Customers have multiple orders so I can't use Max(date)
-
Converting Dataflows
Having a "Convert" button in dataflows would be great for switching from one dataflow to another. For example, a MySQL dataflow to a Magic ETL.
-
Creating a rolling standard deviation within magic ETL
I'd like to create rolling standard deviation within Magic ETL that uses the previous 90 rows (or previous 90 days) as the window. I was able to do this within MySQL before, however, I'd like to do this within Magic ETL. For example if I have 100 rows of data each row should reference the previous 90 rows as the sample…
-
Transforming Support Staff Name Column
Hello, ETL question. For context, I am creating a dashboard for self-service reports on customer satisfaction surveys. Typically I would create a drop down filter card to 'Filter by: Owner', but in this case the owner column contains multiple names: IE: Owner Joe Smith, Bob Smith Cate Smith Cindy Smith, Sam Smith Does…
-
NEW Domo Data Science Resources Guide (help w/ Jupyter, scripting tiles, data science tiles, AutoML)
Hi all, I'm a data scientist at Domo on the Data Science Professional Services Team. We have a NEW Domo Data Science Resources Guide. The guide includes a list of all the help resources Domo has created related to data science products (Jupyter Workspaces, Magic ETL R/Python scripting tiles, Magic ETL data science tiles, &…
-
Joining 3 different datasets to match zip codes
Hi, I have 3 datasets each with a column containing zip codes. I want to use ETL to create a new dataset with 3 columns, all containing the zip codes from each input lined up. Input: Dataset 1 Dataset 2 Dataset 3 1 2 1 2 3 3 3 5 4 4 6 5 Output: D1 D2 D3 1 1 2 2 3 3 3 4 4 5 5 6 Thank you
-
Creating an Artificial downtime for maintenance for only week and work hours
I am trying to calculate downtime within DOMO. The downtime we have currenlty coming from the other system does not put in the factors of weekends and holidays as well as after hours. How can I calculate a correct amount of time between the starttime to endtime while only subtracting within the window. For example if A…
-
Is it possible to make a HTTP request using the Python Script tile of Magic ETL?
We would like to use data of a dataset in the body of a POST request and we were wondering if the Python Script tile in Magic ETL would work for that. We have analyzed but it does not seem to have libraries for that. We have this solution to export the dataset and make the request in a Python script in a Virtual Machine…