API for DataSet tags

ArborRose · 2024-10-02T15:49:08+00:00

There was an error rendering this rich post.

I am having trouble trying to pull values from the Domo API, to identify datasets by "tag". In data center, if I tag some of my datasets as "API", I want to use Python to pull the list of all datasets with that tag.

There's a discussion at this link but it doesn't give a solution:

https://community-forums.domo.com/main/discussion/54450/is-there-an-api-for-viewing-and-manipulating-dataset-tags

I can get a list of my Domo datasets and search by name using "https://api.domo.com/v1/datasets". But the list I need doesn't have a common string in the names. Therefore I need the type or tag. I'm looking for the datasets associated to the code in my Jupyter notebooks.

Find more posts tagged with

Accepted answers

ggenovese

You can send a POST to this endpoint as referenced in the discussion you linked: https://{{your_instance}}.domo.com/api/data/ui/v3/datasources/search

with this post body:

{"entities": ["DATASET"], "filters": [{"filterType": "facetValue", "field": "tag", "value": "<YOUR TAG GOES HERE>*"}], "combineResults": "true", "query": "*", "count": 10, "offset": 0,
"sort": {"isRelevance": "false", "fieldSorts": [{"field": "create_date", "sortOrder": "DESC"}]}}

All comments

MarkSnodgrass

@ArborRose when I inspect the network traffic when I click on a tag in the data center, I see that this is being applied.

This would be against the https://[instance-name].domo.com/api/search/v1/query

Have you tried using this already?

ggenovese

You can send a POST to this endpoint as referenced in the discussion you linked: https://{{your_instance}}.domo.com/api/data/ui/v3/datasources/search

with this post body:

ArborRose

Thank you @MarkSnodgrass and @ggenovese.

I had been searching different endpoints and couldn't find the correct one. @ggenovese , the endpoint you listed was what I needed for my search. After a bit of exhaustive code changes, the following will display a list of the datasets with tag "Your_Tag_Goes_Here". And all the information about the datasets. From there, we can gather the specific fields needed.

My specific use for this is to extract my API dataset list (based on name search for a string I include in all my API calls), and my Jupyter dataset list (based on the tag, using code below) and automate them into a scheduling Gantt chart. I've been hoping Domo would create one.

import requests
import json

# 10.03.24 Author: ArborRose
# This Python code uses the Domo API to retrieve
# datasets based on the "tag". 

# Domo API Credentials (replace with your developer token)
developer_token = 'your_developer_token'

# Set headers for the API call
headers = {
    'X-DOMO-Developer-Token': developer_token,
    'Content-Type': 'application/json'
}

# API URL for fetching datasets and their tags
url = 'https://yourinstance.domo.com/api/data/ui/v3/datasources/search'

# Payload for searching datasets by tag
payload = {
    "entities": ["DATASET"],  # Specify that we want datasets
    "filters": [
        {
            "filterType": "facetValue",
            "field": "tag",
            "value": "Your_Tag_Goes_Here"  # Replace with your tag as needed
        }
    ],
    "combineResults": "true",
    "query": "*",  # Use wildcard to match all
    "count": 500,  # Number of datasets to retrieve
    "offset": 0,  # Starting offset for pagination
    "sort": {
        "isRelevance": "false",
        "fieldSorts": [
            {
                "field": "create_date",
                "sortOrder": "DESC"  # Sort by creation date
            }
        ]
    }
}

# Make the API request to search for datasets
response = requests.post(url, json=payload, headers=headers)

# Check if the request was successful
if response.status_code == 200:
    # Print the full response to understand its structure
    response_data = response.json()
    print(json.dumps(response_data, indent=2))  # Pretty-print the JSON response
else:
    print(f"Failed to fetch datasets: {response.status_code} - {response.text}")

ggenovese

This is great! Thanks for sharing your code! One small enhancement you can make is to use an Account to store your developer token so that it's not in plain text in your workbook

https://domo-support.domo.com/s/article/36004740075?language=en_US#use_accounts

ArborRose

Thanks for the note. When I'm debugging code, I often use a temporary token with a quick expiration. One odd thing I found about the API, it doesn't contain information in the field dataLastUpdated.

ggenovese

In regard to the dataLastUpdated, I'm seeing the same thing on my side. A workaround might be to add last_updated to the filter of your search.

"filters": [
{
"filterType": "facetValue",
"field": "tag",
"value": "Your_Tag_Goes_Here" # Replace with your tag as needed
},
{
"field": "last_updated",
"filterType": "numeric",
"longNumber": 1727755200000,
"operator": "GT"
}
]

Above is your tag search with Last Run: After 10/01/2024 added to it

ArborRose

I assume last updated is when the code was last update, not the data. I need to know the code ran and data was populated via the API it was calling.

jlrosenlof

It's amazing to me that after 2 years nothing has been added to the Dataset API to allow retrieving a Dataset by a tag. We ran into the same problem and the way that we had to solve it was to query the Domo web UI API (as mentioned above and in the linked post), then grab the dataset ID from there and use that to retrieve the dataset(s) using the Dataset API. Seems kinda convoluted.

It seems like it would be a nice enhancement to the Dataset API would be either (or both):

Add an array of tags to the information returned about a dataset.
Allow getting a datasets by tags.

Tags are so useful and integral on the UI side of things and that inevitably follows through to automation API side of things.

Just my 2 cents… Glad you found a solution.

ArborRose

@jlrosenlof - The API has the tag list as an array. In my code above, I'm only pulling based on the tag "Jupyter". But the API does show an array for the datasets that have more than one tag.

ArborRose

I'm still doing a bit of work. But my end goal is to create a blank brick that summarizes my API and Jupyter calls as a custom made Gantt chart. Basically because I couldn't figure out how to do it with a Domo chart. I want to visually see which tasks I have scheduled at specific times to ensure hand-offs and to consolidate the total time of processing.

Using fake data it looks like this: