Is there a way to copy pdp settings between datasets?
I scripted it using PowerShell and the API... All you need to add is the access token and the dataset ids. there are comments in the code from past projects and it is not perfect but it worked for our purpose. We have thousands of PDP rules and the excel pugin bogs down. There is also a PDP utility but you have to create a dataset of PDP rules... This was easier.
function Get-Dataset($datasetId, $Authentication) {#Write-Host $Authentication$headers = @{Accept = 'text/csv'Authorization = $Authentication.access_token}#Write-Host "https://api.domo.com/v1/datasets/$datasetId/data?includeHeader=true&fileName=dump.csv" -Headers $headerstry {$csv = Invoke-RestMethod -Method Get -Uri "https://api.domo.com/v1/datasets/$datasetId/data?includeHeader=true&fileName=dump.csv" -Headers $headers$csv = $csv -replace '\\N', $null$data = ConvertFrom-Csv $csv#Write-Host "must have Successfully Read CSV File"}catch {Write-Host "Failed to Read Dataset CSV File"Write-Host $_.ErrorDetailsWrite-Host $_.ExceptionWrite-Host $_.FullyQualifiedErrorIdStart-Sleep -Seconds 10}return $data}
function Get-Dataset-Object($datasetId, $Authentication) {#Write-Host $Authentication$NeedData = $trueWhile( $NeedData -eq $true -and $Cnct_Cntr -lt 100){try {$headers = @{Accept = 'application/json'Authorization = $Authentication.access_token}#Write-Host "https://api.domo.com/v1/datasets/$datasetId" -Headers $headers$data = Invoke-RestMethod -Method Get -Uri "https://api.domo.com/v1/datasets/$datasetId" -Headers $headers#Write-Host "must have Successfully Read CSV File"$Cnct_Cntr++$NeedData = $false}catch {Write-Host "Failed to get dataset object"Write-Host $_.ErrorDetailsWrite-Host $_.ExceptionWrite-Host $_.FullyQualifiedErrorId$Cnct_Cntr++Write-Host $Cnct_CntrStart-Sleep -Seconds 10}}return $data}
function Get-Dataset-PDP($datasetId, $Authentication) {#Write-Host $Authentication$NeedData = $trueWhile( $NeedData -eq $true -and $Cnct_Cntr -lt 100){try {$headers = @{Accept = 'application/json'Authorization = $Authentication.access_token}#Write-Host "https://api.domo.com/v1/datasets/$datasetId" -Headers $headers$data = Invoke-RestMethod -Method Get -Uri "https://api.domo.com/v1/datasets/$datasetId/policies" -Headers $headers#Write-Host "must have Successfully Read CSV File"$Cnct_Cntr++$NeedData = $false}catch {Write-Host "Failed to get dataset object"Write-Host $_.ErrorDetailsWrite-Host $_.ExceptionWrite-Host $_.FullyQualifiedErrorId$Cnct_Cntr++Write-Host $Cnct_CntrStart-Sleep -Seconds 10}}return $data}
function Get-Authorization($clientID, $clientSecret) {$basicAuth = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $clientID, $clientSecret)));$authenticationHeader = @{Authorization = ("Basic {0}" -f $basicAuth)}$NeedToken = $true$Cnct_Cntr = 0
#keep trying to get a tokenWhile( $NeedToken -eq $true -and $Cnct_Cntr -lt 100){try {$data = Invoke-RestMethod -Method Post -Uri "https://api.domo.com/oauth/token?grant_type=client_credentials&scope=data%20user" -Headers $authenticationHeader# $data = Invoke-RestMethod -Method Post -Uri "https://api.domo.com/oauth/token?grant_type=client_credentials&scope=full" -Headers $authenticationHeader$data.access_token = "bearer $($data.access_token)"# Write-Host "Got Token"$Cnct_Cntr++$NeedToken = $false}catch {#$ErrorActionPreference = Stop #this does not recognize "Stop"Write-Host "Failed to get Token"Write-Host $_.ErrorDetailsWrite-Host $_.ExceptionWrite-Host $_.FullyQualifiedErrorId$Cnct_Cntr++Write-Host $Cnct_CntrStart-Sleep -Seconds 10}}return $data}
function Send-ToEmail([string]$email, [string]$body){
$message = new-object Net.Mail.MailMessage;$message.From = "Domo@daltile.com";$message.To.Add($email);$message.Subject = "Error Running Vistex jobs Script";$message.Body = $body;$smtp = new-object Net.Mail.SmtpClient("smtpmail.mohawkind.com");$smtp.send($message);write-host "Mail Sent" ;}
Write-Host "go get auth token"$Authentication = Get-Authorization -clientID $apiKey -clientSecret $apiTokenWrite-Host "have token"#Write-Host $Authentication#$MetData = Get-Dataset -datasetId $datasetmetadata -Authentication $Authentication
$output = ''$haserr = $true$Have_PDP = $false#get the list of WB jobswhile(!$Have_PDP){try{$Authentication = Get-Authorization -clientID $apiKey -clientSecret $apiToken$DS_PDP = Get-Dataset-PDP -datasetId $PDPSourceDatasetID -Authentication $Authentication$Have_PDP = $true}catch{Write-Host "Failed to get dataset list of PDP rules"Write-Host $_.ErrorDetailsWrite-Host $_.ExceptionWrite-Host $_.FullyQualifiedErrorIdStart-Sleep -Seconds 10}}# Cycle through the list of jobs ensuring they run successfully.
$PDPs = $DS_PDP.Where{$_."name" -like "z_rep_*" -or $_."name" -like "z_ssc_*" } #| Sort-Object -Property $_."Execution Order"| Sort-Object -Property $_."Execution Order"foreach($PDP in $PDPs){$haserr = $true$output = ''$msg = ''$Count = 0Try{$hdrs = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"$hdrs.Add("Content-Type", "application/json")$hdrs.Add("Accept", 'application/json')$hdrs.Add("Authorization", $Authentication.access_token)$filter = $PDP.filters$name = $PDP.name$type = $PDP.type$users = $PDP.users$groups = $PDP.groups
$i = 0$pdp_rule = '{ "name": "'+$name+'",'$fString = '"filters": ['foreach($f in $PDP.filters){$fString += '{ "column": "'+$f.column+'", "values": [ "'+$f.values+'" ], "operator": "'+$f.operator+'" }'$i++if($i -lt $PDP.filters.count){$fString += ','}}
$pdp_rule += $fString +' ], "groups": [ "'+$groups+'" ] }'# {# "name": "z_ssc_4000",# "filters": [ {# "column": "business_area_number",# "values": [ "4000" ],# "operator": "EQUALS"# } ],# "groups": [ 612003576 ]#}
$post = Invoke-WebRequest -Method POST -Uri "https://api.domo.com/v1/datasets/$DestinationDatasetId/policies" -UseBasicParsing -Body $pdp_rule -Headers $hdrs -ContentType 'application/json'
Write-Host "updated $name in Proex dataset PDP "}catch {Write-Host "Failed to add $email to Domo"Write-Host $_.ErrorDetailsWrite-Host $_.ExceptionWrite-Host $_.FullyQualifiedErrorId}}#end foreach job
$now = Get-Date -Format gWrite-Host "$now Done."
There isn't a process currently place through the interface that lets you do this. I think there's a post on the Ideas Exchange for it, though. Ideally, I think, there would be a list of PDPs stored independent of datasets, and you could apply a group of policies to datasets at the click of a button.
That said, we're using a not-so-official software application that Domo skunkworks wrote that lets us apply policies across different datasets using a config file hosted in Domo via a webform dataset. It's saved us a ton of time and serves the same purpose. You should inquire with your success manager or rep about that.
I was trying to search for this ap but couldn't. Is there anyway that I can access and use it to copy over PDP to differnt data set?
I'm sure this application isn't available to download. You'll have to ask Domo directly.
I'd also watch out in the coming months to see if something comes into the product. I wouldn't be surprised to see something at Domopalooza, for example.
I'm revisiting this because I've continued to builed datasets that have the exact same PDP settings as previous datasets, and I still don't see a way to copy the settings from one dataset to another.
Does this exist or do I have to continue recreating the wheel on every dataset?
Have you tried the Excel plugin for DOMO, this allows you to apply PDP security to any dataset you want and is a fast and easy way to apply PDP policies that are exact copies to other datasets.
Just change the DatasetID and apply and your good.
This is also very easy to make slight modifications to PDP's that are close but not the same.
It is also an easy way to manage other components such as user accounts etc.
Randy
I will request the admin version of that.
Thanks.
You should be able to download it from the Admin/Tools Download section.
The admin one requires a special request, but I did get. It's not quite the elegant solution I was hoping for within DOMO, but it will work for when I have to add bulk permissions.
Hi @Randyb , can you provide some more information on how you accomplished this using the Excel plugin? I've got a massive PDP update project I need to undertake, and this sounds like a great option for us. Are you exporting a specific dataset into Excel then re-uploading with changes, or something else?
Thanks!
-Russell
@rhollander There are a couple of ways you can do this, you can download the excel plugin which then allows you to pull the data down along with the different PDP profiles. You can then build essentially a user list for a specific PDPPolicy and upload it.
https://knowledge.domo.com/Connect/Personalized_Data_Permissions_(PDP)/PDP_Policy_Autocreation
@rhollander , take a look at this utility, may help you automate the process