Trying to append data from databricks
Hello,
My requirement is to pull the data - run a query 90 columns from databricks table for 12 month which is around 2M records of data and append it to the existing dataset via API
Since its the large volume of data, I'm getting error as '"status":400,"statusReason":"Bad Request","message":"Too many non-consecutive part ids. Make consecutive or upload multiple data versions'.
I then tried to run the query by month and append it to the dataset using
domo.ds_create('datasetname')
domo.get('datasetid')
Now I get error 'Python int too large to convert to C long'.
Can someone help me here or provide the proper syntax to append the data daily to the existing dataset using python script that I can run from databricks?
Best Answer
-
Have you tried using the limit and offset parameters in your GET request to extract the data from Domo paginated?
**Was this post helpful? Click Agree or Like below**
**Did this solve your problem? Accept it as a solution!**0
Answers
-
Have you tried using the limit and offset parameters in your GET request to extract the data from Domo paginated?
**Was this post helpful? Click Agree or Like below**
**Did this solve your problem? Accept it as a solution!**0 -
Can you give me more details about it please?
0
Categories
- All Categories
- 2K Product Ideas
- 2K Ideas Exchange
- 1.6K Connect
- 1.3K Connectors
- 311 Workbench
- 6 Cloud Amplifier
- 9 Federated
- 3.8K Transform
- 657 Datasets
- 115 SQL DataFlows
- 2.2K Magic ETL
- 815 Beast Mode
- 3.3K Visualize
- 2.5K Charting
- 81 App Studio
- 45 Variables
- 775 Automate
- 190 Apps
- 481 APIs & Domo Developer
- 81 Workflows
- 23 Code Engine
- 40 AI and Machine Learning
- 20 AI Chat
- 1 AI Playground
- 1 AI Projects and Models
- 18 Jupyter Workspaces
- 410 Distribute
- 120 Domo Everywhere
- 280 Scheduled Reports
- 10 Software Integrations
- 144 Manage
- 140 Governance & Security
- 8 Domo Community Gallery
- 48 Product Releases
- 12 Domo University
- 5.4K Community Forums
- 41 Getting Started
- 31 Community Member Introductions
- 114 Community Announcements
- 4.8K Archive