Filter Data Using Domo API for Python

Hi,

I am currently trying to pull data from Domo Dataset into a csv using the python API.

I am able to pull the complete dataset but would like to filter the data based on certain conditions.
Can anyone help with me with.

This is the code that i have got to pull the Data from the Dataset.


import logging
from pydomo import Domo

# Build an SDK configuration
client_id = ''
client_secret = ''
api_host = 'api.domo.com'
logger_name = ''
log_level = logging.INFO

# Create an instance of the SDK Client
domo = Domo(client_id, client_secret, api_host=api_host, logger_name=logger_name, log_level=log_level)

csv_file_path = r'/abc.csv'
include_csv_header = True
csv_file = domo.datasets.data_export_to_file('', csv_file_path, include_csv_header)
csv_file.close()

Best Answer

  • GrantSmith
    GrantSmith Coach
    Answer ✓

    @user02319 

     

    Correct, you'd need to read the data into a dataframe but you're essentially doing that right now anyway with your dataset pull - it's pulling all of the data into the file object you'd just need to convert it to a dataframe.  I don't see a way to filter the dataset using the Python SDK currently. It's just pulling the entire dataset you specify.

    **Was this post helpful? Click Agree or Like below**
    **Did this solve your problem? Accept it as a solution!**

Answers

  • Hi @user02319 

     

    A highly popular python package for analytics and processing data is called pandas (https://pandas.pydata.org). It allows you to read CSV files (and other sources) into a DataFrame (essentially a data table) with columns and rows. This allows you to then filter  specific rows or columns based on specific criteria and then export the data back out using the to_csv method.

     

    Here's a sample webpage on filtering with pandas examples: https://www.listendata.com/2019/07/how-to-filter-pandas-dataframe.html

    **Was this post helpful? Click Agree or Like below**
    **Did this solve your problem? Accept it as a solution!**
  • Hi @user02319 

     

    A highly popular python package for analytics and processing data is called pandas (https://pandas.pydata.org). It allows you to read CSV files (and other sources) into a DataFrame (essentially a data table) with columns and rows. This allows you to then filter  specific rows or columns based on specific criteria and then export the data back out using the to_csv method.

     

    Here's a sample webpage on filtering with pandas examples: https://www.listendata.com/2019/07/how-to-filter-pandas-dataframe.html

    **Was this post helpful? Click Agree or Like below**
    **Did this solve your problem? Accept it as a solution!**
  • Hi @user02319 

    A highly popular python package for analytics and processing data is called pandas. It allows you to read CSV files (and other sources) into a DataFrame (essentially a data table) with columns and rows. This allows you to then filter  specific rows or columns based on specific criteria and then export the data back out using the to_csv method.

     

    **Was this post helpful? Click Agree or Like below**
    **Did this solve your problem? Accept it as a solution!**
  • Hey @GrantSmith ,

     

    Thanks for the reply. I had thought about using pandas dataframes to filter the data. But that would mean i would need to pull all of the data in the dataframe first then filter it right??

    If yes then i am looking for a solution where i can filter the data directly at the domo end itself

  • GrantSmith
    GrantSmith Coach
    Answer ✓

    @user02319 

     

    Correct, you'd need to read the data into a dataframe but you're essentially doing that right now anyway with your dataset pull - it's pulling all of the data into the file object you'd just need to convert it to a dataframe.  I don't see a way to filter the dataset using the Python SDK currently. It's just pulling the entire dataset you specify.

    **Was this post helpful? Click Agree or Like below**
    **Did this solve your problem? Accept it as a solution!**
  • Hey @GrantSmith ,

     

    Got it.  Will try it out with pandas and see. Thanks for the help.

This discussion has been closed.