Copy Beastmodes to New Dataset

Benjamin

I have a case where I have a dataset that has to large of a date range that is causing a query to run for a long period of time.  My solution is to create an archived dataset and a current dataset and then use a SQL join within DOMO to stack them.    This solution has worked well to create a dataset I want, however, I have many beastmodes on the original dataset and I want to move them to this new one created by the union.

 

My plan is to review multiple other datasets that are in this similar situation so that I can relieve resources on our servers for the queries.

 

Is there anyway to copy all beastmodes from one dataset into another?

jaeW_at_Onyx

@Benjamin ,

 

splitting data into Current and Archive might be a solution... but are you sure the problem will be resolved by splitting the data into two sets?  obviously i can't see your ETL, but it sounds like you're just moving your execution time bottleneck to another place... how are you building up your archive?  b/c as that dataset grows, ETL execution time is going to just increase, unless you're not using ETL tools.

 

AFAIk, unless you query APIs directly, there isn't a way to copy beast modes from one dataset to another (if the beast mode is saved to the dataset).  if you move a card from one dataset to another AND the beast modes referenced in the card are saved to the card, THEN they'll copy over assuming the column names are the same.

 

There are new tools like the Beast Mode Manager (a beta product, talk to your CSM) which will make managing Beast modes easier, but AFAIK in this current release, there isn't a way to copy from one dataset to another.

 

you could use the Domo Governance Beast Modes dataset to identify all the beast modes saved to a dataset, and then just basically create a checklist of every BM you need to recreate on the new dataset.

 

hope that helps!

jaeW_at_Onyx

@Benjamin ,

 

splitting data into Current and Archive might be a solution... but are you sure the problem will be resolved by splitting the data into two sets?  obviously i can't see your ETL, but it sounds like you're just moving your execution time bottleneck to another place... how are you building up your archive?  b/c as that dataset grows, ETL execution time is going to just increase, unless you're not using ETL tools.

 

AFAIk, unless you query APIs directly, there isn't a way to copy beast modes from one dataset to another (if the beast mode is saved to the dataset).  if you move a card from one dataset to another AND the beast modes referenced in the card are saved to the card, THEN they'll copy over assuming the column names are the same.

 

There are new tools like the Beast Mode Manager (a beta product, talk to your CSM) which will make managing Beast modes easier, but AFAIK in this current release, there isn't a way to copy from one dataset to another.

 

you could use the Domo Governance Beast Modes dataset to identify all the beast modes saved to a dataset, and then just basically create a checklist of every BM you need to recreate on the new dataset.

 

hope that helps!

Benjamin

I appreciate your response.

 

Let me give you a little more background and see your thoughts.

 

Originally I had this dataset setup to pull from 2017 through today and it would take approx 15 minutes to run.  My end goal was to shorten the amount of time it would take to update this dataset.  I am pulling this data into DOMO through workbench running a SQL query against our DB with an ODBC connection.

 

My approach was to create an archive dataset and a current dataset.  The thought is that I can make the archive run from 2017 to current year and current dataset run 2020 only.  I could only run the current dataset and append/join to the archive.

 

In testing the archive dataset takes about 8 minutes to run and the current about 1.5 minutes.  

 

I don't know the best method of stacking these in DOMO so I tried two.  The first I tried was through dataflows SQL and used a UNION for the two datasets.  This takes approx 25 minutes to run so it was not a better solutoin.

 

My second try was through dataflows magic ETL and I used the append function.  Surprisingly this only took 8 minutes.  So now I've reduced the overall update time from 15 minutes to 8.   

 

I'm not sure what additional methods I should try that may be faster?

Benjamin

I just tried one additional test and this time I used the Domo Blend function with the Add Rows feature and that appeared to be almost instantaneous.  

 

If there is no downside of this being a datafusion blend then this appears to be the best method?  It would cut down my update time from 15 minutes to approx 2 minutes.

GrantSmith

@Benjamin - DataFusions are a lot quicker than an ETL. I try to use them first if at all possible. The one current downside is that you can't use the fustion dataset as an input into a future ETL (Magic or MySQL) but I believe there might be a beta which will allow just that (along with DataSet Views / another beta).

Aditya_Jain

Hi @Benjamin 

You can do this by going to the beast mode manager.

Click on Data Center, on the left, lookout for Beas Mode manager

On the Beast Mode manager page, on the left pane, just type in your beast mode name, select the beast mode, and on the right hand side wrench icon, select duplicate to dataset.

Choose your target dataset.

This way you can copy beast mode to new datasets.