Filter Data Using Domo API for Python

user02319

Hi,

I am currently trying to pull data from Domo Dataset into a csv using the python API.

I am able to pull the complete dataset but would like to filter the data based on certain conditions.
Can anyone help with me with.

This is the code that i have got to pull the Data from the Dataset.

import logging
from pydomo import Domo

# Build an SDK configuration
client_id = ''
client_secret = ''
api_host = 'api.domo.com'
logger_name = ''
log_level = logging.INFO

# Create an instance of the SDK Client
domo = Domo(client_id, client_secret, api_host=api_host, logger_name=logger_name, log_level=log_level)

csv_file_path = r'/abc.csv'
include_csv_header = True
csv_file = domo.datasets.data_export_to_file('', csv_file_path, include_csv_header)
csv_file.close()

GrantSmith

@user02319 

 

Correct, you'd need to read the data into a dataframe but you're essentially doing that right now anyway with your dataset pull - it's pulling all of the data into the file object you'd just need to convert it to a dataframe.  I don't see a way to filter the dataset using the Python SDK currently. It's just pulling the entire dataset you specify.

GrantSmith

Hi @user02319 

 

A highly popular python package for analytics and processing data is called pandas (https://pandas.pydata.org). It allows you to read CSV files (and other sources) into a DataFrame (essentially a data table) with columns and rows. This allows you to then filter  specific rows or columns based on specific criteria and then export the data back out using the to_csv method.

 

Here's a sample webpage on filtering with pandas examples: https://www.listendata.com/2019/07/how-to-filter-pandas-dataframe.html

GrantSmith

Hi @user02319 

 

A highly popular python package for analytics and processing data is called pandas (https://pandas.pydata.org). It allows you to read CSV files (and other sources) into a DataFrame (essentially a data table) with columns and rows. This allows you to then filter  specific rows or columns based on specific criteria and then export the data back out using the to_csv method.

 

Here's a sample webpage on filtering with pandas examples: https://www.listendata.com/2019/07/how-to-filter-pandas-dataframe.html

GrantSmith

Hi @user02319 

A highly popular python package for analytics and processing data is called pandas. It allows you to read CSV files (and other sources) into a DataFrame (essentially a data table) with columns and rows. This allows you to then filter  specific rows or columns based on specific criteria and then export the data back out using the to_csv method.

 

user02319

Hey @GrantSmith ,

 

Thanks for the reply. I had thought about using pandas dataframes to filter the data. But that would mean i would need to pull all of the data in the dataframe first then filter it right??

If yes then i am looking for a solution where i can filter the data directly at the domo end itself

GrantSmith

@user02319 

 

Correct, you'd need to read the data into a dataframe but you're essentially doing that right now anyway with your dataset pull - it's pulling all of the data into the file object you'd just need to convert it to a dataframe.  I don't see a way to filter the dataset using the Python SDK currently. It's just pulling the entire dataset you specify.

user02319

Hey @GrantSmith ,

 

Got it.  Will try it out with pandas and see. Thanks for the help.