API Partitioning
Hi all, so I want to use the Domo API's to do partitioning. That is, I want to send data daily, and I do not want the data there to be overwritten. I am trying this but I think it is being overwritten:
https://api.domo.com/v1/datasets/{dataset_id}/data?updateMethod=PARTITION
Is this the correct way to upload via partition?
Also, where can I find good information and documentation on this?
Best Answer
-
If I grasp the question correctly…
If you are setting up an API, you want to go to the Domo AppStore and find the proper API connector. Type JSON in the search. Something like the JSON no code connector. In the connector you will input your settings like the URL you show. "updateMethod" looks like a parameter. You should find fields for entering parameters for header, body, or date.
Complete the form. Set your data structure and schedule. It will return a dataset. Partitioning is normally done using an ETL after you have the dataset from the API.** Was this post helpful? Click Agree or Like below. **
** Did this solve your problem? Accept it as a solution! **0
Answers
-
Hi @mroker, could you further clarify the issue you are trying to solve for?
0 -
If I grasp the question correctly…
If you are setting up an API, you want to go to the Domo AppStore and find the proper API connector. Type JSON in the search. Something like the JSON no code connector. In the connector you will input your settings like the URL you show. "updateMethod" looks like a parameter. You should find fields for entering parameters for header, body, or date.
Complete the form. Set your data structure and schedule. It will return a dataset. Partitioning is normally done using an ETL after you have the dataset from the API.** Was this post helpful? Click Agree or Like below. **
** Did this solve your problem? Accept it as a solution! **0 -
Thanks @ArborRose . So yes, I didn't know it was that easy in the ETL. Basically I wanted to use the API endpoint to partition the way Workbench does. I wanted to push data into Domo and if there was overlapping data, for it not to be duplicated. Also, if there was additional data, to append. So now what I am doing to solve that is to just use the replace on the API endpoint and I've created a ETL where the output is using a partition. It works perfectly.
1 -
@mroker - Glad to hear it's working.
After you get your API configured, I suggest making sure the data structure matches your data. The JSON connectors will try to guess at what your data is. If your field holds a date, make sure its a date. Not a string, not datetime…a date. Make sure joining identifiers come across as string or whatever you need. Otherwise, you'll kill yourself later trying to convert all those fields to cast them into something they should have been from the start.
With the No Code oAuth Connector, you can adjust the schema after its predicted the field types.** Was this post helpful? Click Agree or Like below. **
** Did this solve your problem? Accept it as a solution! **0
Categories
- All Categories
- 1.5K Product Ideas
- 1.5K Ideas Exchange
- 1.4K Connect
- 1.1K Connectors
- 278 Workbench
- 4 Cloud Amplifier
- 4 Federated
- 2.7K Transform
- 89 SQL DataFlows
- 560 Datasets
- 2K Magic ETL
- 3.3K Visualize
- 2.3K Charting
- 575 Beast Mode
- 13 App Studio
- 28 Variables
- 584 Automate
- 142 Apps
- 415 APIs & Domo Developer
- 26 Workflows
- 1 DomoAI
- 28 Predict
- 12 Jupyter Workspaces
- 16 R & Python Tiles
- 356 Distribute
- 95 Domo Everywhere
- 259 Scheduled Reports
- 2 Software Integrations
- 92 Manage
- 89 Governance & Security
- 9 Product Release Questions
- Community Forums
- 42 Getting Started
- 28 Community Member Introductions
- 89 Community Announcements
- 4.8K Archive