Dataset not updating.
Can someone help me with this issue?
I'm trying to pull the data from databricks (using python script) into domo dataset.
First I'm trying to create the dataset for a single date using
#domo.ds_create(df2,'datasetname')
then I'm trying to update the same dataset daily. the below code worked till yesterday and its only replacing it from today onwards. not sure what went wrong.
import pandas as pd
import numpy as np
df = spark.sql("Select a, b,c from table a where date > 2023-08-01") — giving a example query
df2 = df.toPandas()
domo.ds_get('datasetid')
Update the Domo datasetdomo.ds_update('datasetid', df_up = df2)
Retrieve dataset from Domodataset_id = 'datasetid'
try:
original_data = domo.ds_get(dataset_id)
except Exception as error:
print(f'Error retrieving dataset: {error}')
# Handle error as needed
...
if original_data.empty:
print('No data retrieved from Domo')
# Handle empty data as needed
...
else:
# Append data to itself
updated_data = original_data.append(original_data).reset_index(drop=True)
# Update dataset in Domo
try:
domo.ds_update(dataset_id, df_up=updated_data)
except Exception as error:
print(f'Error updating dataset: {error}')
# Handle error as needed
Answers
-
Have you validated the data you're getting back from your SQL query? Is it returning all of the data you're expecting?
**Was this post helpful? Click Agree or Like below**
**Did this solve your problem? Accept it as a solution!**0
Categories
- All Categories
- 1.8K Product Ideas
- 1.8K Ideas Exchange
- 1.6K Connect
- 1.2K Connectors
- 302 Workbench
- 6 Cloud Amplifier
- 9 Federated
- 2.9K Transform
- 104 SQL DataFlows
- 633 Datasets
- 2.2K Magic ETL
- 3.9K Visualize
- 2.5K Charting
- 760 Beast Mode
- 62 App Studio
- 42 Variables
- 699 Automate
- 181 Apps
- 457 APIs & Domo Developer
- 51 Workflows
- 10 DomoAI
- 38 Predict
- 16 Jupyter Workspaces
- 22 R & Python Tiles
- 401 Distribute
- 116 Domo Everywhere
- 277 Scheduled Reports
- 8 Software Integrations
- 130 Manage
- 127 Governance & Security
- 8 Domo Community Gallery
- 38 Product Releases
- 12 Domo University
- 5.4K Community Forums
- 40 Getting Started
- 30 Community Member Introductions
- 111 Community Announcements
- 4.8K Archive