Magic ETL

Hi, I am trying to join a table on my DataSet, using the left join function. My original DataSet contains only 118k rows, and the other table contains 1405 rows. I thought that the output DataSet will stay 118k rows, and just have addition columns on it. However, the output DataSet is 4.6M which is way more then I…

I have a column that contains values that are 4 digits. I want to divide them up in to 2 separate columns. The first 2 number are the % of the discount, and the last 2 are the length Ideally it would be the % and contain 30, 40, 35, 40 ,etc... Then Length would have the last for digits like 02, 02, 03, 02, 04, etc.. I am…

I have a date column that is in this format: The format of the date is fine, but what I am trying to do is get rid of everything including the 'T' and to the right, so that all that is left would be 2021-06-27. If a solution within the ETL is possible that would be great.

In Magic ETL, a Select Columns tile was inserted just prior to the Output Dataset tile in order to reorder columns. The columns in the resulting Output Dataset were not in the order specified in the Select Columns tile. Is there something else that needs to be considered so that the column order is retained in the Output…

Hello, Experts, I am humbly reaching out for some help. I have a vendor sending a snapshot of a database table every day and not sending me diffs. I am trying to determine the diffs. The connector type is SFTP CSV. The file they send daily is overwritten daily. I currently have the data connector appending the data from…

Hello, Domosapiens, I currently have a data flow that is taking 14 min or greater to run. The design is as such: MySQL connector that first runs once to pull in all historical data (7.8M rows). Ex full historical pull query: select * from table; change dataset settings to merge method by changing the query to select * from…

I just set up a connection for a dataset that doesn't have any fixed 'date' column. The only date column it will have will be the batch_last_run date. On a nightly basis this dataset will append and start to build history. How can I add a new date column in a format such as 'Jul 1, 2021', so that the new data and old data…

I have a column with multiple categories values, I want to convert this categories values to column labels. How do I do this. In the screenshot attached below I want to convert values of column '123 1' to column labels.

How do i trim out an email address from a field with a string? Example of Field: Yes, we accept this current pricing and would like to proceed with this order. Thank you, Bob Dole Lab Research Analyst Poss Lab—Dole University bob.dole@some.edu Thanks

I currently have this code: categorizing some leads. Select `Year-Month` ,`Lead Source` ,`Product Line` ,`Product Models` ,`Country` ,`Early Stage No Movement` ,`Stage 0 to Early` ,`Early to Lost` ,`Early to Won` ,`Early to Late` ,`Late to Lost` ,`Late to Won` ,Sum(`Early to Lost`+`Early to Won`+`Early to Late`) AS 'Total…

Hi, I have a situation where a column my have different rows of the same values, but the values are in different orders and therefore they are different buckets in cards, when they're actually the "same" thing. And example of this is in A1:B3 of my attached image, where both rows under the BH_TAT.Antigens column have the…

Attached are 2 different samples of data. Both are in similar format, and have plenty of same columns. The price list is where the bulk of what I am going to use will come from. It is the full inventory for a site. the total number of units, number of occupied, number of vacant etc... It is rolled up in to a single row for…

I have a column called 'description' which can contain multiple elements. One of them is 'Order #1234567890' (or other digits). I want to create a new column that just extracts the numbers in SQL or Magic ETL (so that I can do Beast Modes etc. on it afterwards). How can I create this column? Numbers may exist in other…

I am using the Filter Rows aspect of Magic ETL and having trouble filtering if the text filed ins not one word. For example if I create a filter that says ColumnA equals TESTWORD - I get rows where the columns had the value TESTWORD. If I create a filter that tries to filter ColumnA equals TEST WORD - (space between test…

Hi, I am creating a new variable using the add formula icon where I am creating time period bins that reflects the tenure of the consultant in reference to today. This is the formula I created: case when CURRENT_DATE() -`CD (Start)` < 90 then '3 Months' when CURRENT_DATE() -`CD (Start)` >= 90 and CURRENT_DATE() -`CD…

I am trying to collapse my data set into a single row for each date, publication and product and get the error message "Action outputs to an invalid number (0) of actions" My data is structured as per below:- Date Product Publication Channel Revenue 1/1 Display Publication A OMP £1000 Where Date, product, publication and…

I think I just discovered that if a magic ETL is configured to run "only when datasets are updated", and has multiple input datasets, it will start running as soon as the smallest/quicket input dataset has been updated. If a larger/slower input dataset finishes updating before the ETL finishes running, the ETL will not run…

When building an ETL, how can I view the whole output of an action within the ETL? After running the preview I can only view 100 rows. Example: 1. I join two datasets (that are created within the ETL) to one without knowing if the join-columns are unique. I want to view the datasets that go into the Join action and the…

Hi, I am using Magic ETL for the first time and I am using two datasets and a left join to combine the two. I only want to left join one column from the 2nd table, but I cannot find where I can select for that in Magic ETL

Because of how fast/easy ETL flows are, I prefer to use them for most flows. My problem is that sometimes I hit the following error, "Error joining data. The left input cannot include over 10,000 duplicates. Please switch your inputs, group your data, or remove duplicates before joining". The Dojo suggests that this error…

Once you are logged into the Help Center, you can move seamlessly between Domo University Training, Knowledge Base, Community and Support (using the head navigation menu) . Knowledge Base articles are a great resource to answer many general questions. KB articles answering the most common Beast Mode/ETL questions: * Beast…

* If you are struggling with long run times * Join big datasets at Domo Fusion speed and use them in ETL * Consistent failure of ETL after mulitple hours * Want to work on a subset of a big dataset * Create Views in Domo

When I try to extract the _BATCH_LAST_RUN_ from a MySQL dataset with 106M rows, the ETL takes 3h20min. Is there a way to extract only one row from the dataset. For the moment, I have to wait until the full 106M rows dataset is load in both Magic ETL and SQL. 

Hi Guys, Is there a way to change the column name in a dataset? I'm using Informatica to load in my datasets into Domo. I know you can create an ETL flow but the dilemma I'm facing is: I have a dataset (Dataset_A1) which had 4 columns, account | record | id | amount_given I created a dataflow to edit the column names &…