-
Creating a separate column in transforming data using magic ETL
I am using formula function to add a column that gives out values based on certain condition. I am using case condition for validating the conditions, and I want it to be grouped by Req Id field. because req Id field contains multiple req Ids and have duplication. Idea behind grouping by is that if any of that condition is…
-
Convert DATETIME (UTC) to specific timezone
Hello, Domosapiens! I have an interesting quandry. I have a data flow (using MagicETL - should I be using something else?) that I am trying to build and convert a date that my MySQL data connector is bringing over. The datetime column comes from an orders table and is called posted_date and it brings the data over as UTC…
-
Improve Dataflow Efficiency
Hello, Domosapiens, I currently have a data flow that is taking 14 min or greater to run. The design is as such: MySQL connector that first runs once to pull in all historical data (7.8M rows). Ex full historical pull query: select * from table; change dataset settings to merge method by changing the query to select * from…
-
DataFlow Transform (i.e. Action) Types in API JSON data
My question: What factors determine the type of a dataflow transform in the api JSON data (e.g. GenerateTableAction, SQL, SqlAction)? Background: I am using DOMO's command line tool to get JSON structured metadata about a DataFlow. Specifically, I am using the list-dataflow command: Putting in the DataFlow id and a…
-
Regards to transforming data prior to joining with another data set.
Attached are 2 different samples of data. Both are in similar format, and have plenty of same columns. The price list is where the bulk of what I am going to use will come from. It is the full inventory for a site. the total number of units, number of occupied, number of vacant etc... It is rolled up in to a single row for…
-
Is there an alternative of distinct on for redshift?
Hey, I am trying to build a dataflow using redshift but distinct on is not supported. It's a simple query: (SELECT distinct on ("id") var_list.* from "indicator" as ind order by "id", "rank" desc) Any suggestions for workarounds?
-
Do not Generate Output Table in transformation not available
Hello, I'm looking how to not Generate an Output Table in a transformation ? I did not have this option neither in the transformation neither the dataflow. In lot of tutorial, I saw that a pop up appear with : MagicETL, MySQL and Redshift and then in the editor there is a dropdown list with thoses 3 same options. I did not…
-
Pivoting data in data flows (Year over year || week over week)
Hi, I have account information on one of my data sets like this: Registrations: Name Date_of_Registration Daniel 2015-12-01 John 2015-10-22 Peter 2014-12-01 I want to use dataflows in order to pivot this data in two ways: #1 New registrations per year 2014 2015 1 2 #2 New registrations per week Week 2014 2015 10 0 1 12 1 1…