Ability to Edit Virtual Dataset Jobs

alexis_lundberg

The ability to edit virtual dataset jobs would help a lot when working between production and sandbox environments. We would like to group our virtual dataset jobs by data domain, to help make sure we don't end up with duplicate datasets in our production environment. However if we create a new dataset we currently cannot add it to an existing virtual dataset job, and this increases our risk of duplication.

MichelleCCameronMYO

Absolutely - duplication is daunting.

brycec

I was playing around in the Product APIs and found that editing Virtual DataSet jobs is possible via the API, just not the UI.
1. Get the applications (different toolkits):

GET https://<your_instance>.domo.com/api/executor/v1/applications

2. Use the application ID for the Virtual DataSets to get all the jobs:

GET https://<your_instance>.domo.com/api/executor/v2/applications/<application_id>/jobs?limit=100&offset=0

3. Get the job you want to update:
GET https://<your_instance>.domo.com/api/executor/v1/applications/<application_id>/jobs/<job_id>

4. Update the job:

PUT https://<your_instance.domo.com/api/executor/v1/applications/<application_id>/jobs/<job_id>

For the body, use the response from the get job endpoint, but remove the properties executionState and shareState, and add or modify the objects in the array executionPayload.sharingRequests. You can find the filterGroupId by using the following endpoint:
GET https://<your_instance>.domo.com/api/query/v1/data-control/<dataset_id>/filter-groups?options=load_associations,include_open_policy,load_filters,sort

Authentication to the Product APIs is explained at the bottom of this page: https://developer.domo.com/portal/8ba9aedad3679-ap-is

cadellfalconer

@alexis_lundberg - @MichelleCCameronMYO - @brycec
Apologies for what might seem like silly questions, but I'm trying to get a solid understanding of our customers and their usage of virtualized datasets, along with any gaps they might have:

Do you use Governance Toolkit for your virtualized datasets? If not, what method do you use set them up? Brycec, I notice you're leveraging APIs, is that the only way you manage virtualized datasets?

And for a thought experiment - If you imagine you're a Domo employee, and were tasked with creating the virutalized datasets feature - how would you design them? What would the user experience be? What are the biggest customer problems you would solve with virtualized datasets and how? Where would they live? Or any other things that jump out to you as you think of this.

brycec

@cadellfalconer

I do use governance toolkit, as well as the APIs. The toolkit is limited by the inability to edit a job, and that a new job must be created every time you want to create virtualized DataSets.

It would be very nice if the toolkit could be an automation, much like how other toolkit jobs work. For example, I would create a job that would create virtualized DataSets to all DataSets with "|PROD|" at the end of their name to our sandbox instance, and have it run once a day. In a way, it might also be nice to create, view, and manage virtualized DataSets from the DataSet record in the production instance.