API Partitioning

mroker · 2024-02-13T21:06:22+00:00

There was an error rendering this rich post.

Hi all, so I want to use the Domo API's to do partitioning. That is, I want to send data daily, and I do not want the data there to be overwritten. I am trying this but I think it is being overwritten:

https://api.domo.com/v1/datasets/{dataset_id}/data?updateMethod=PARTITION

Is this the correct way to upload via partition?

Also, where can I find good information and documentation on this?

Find more posts tagged with

API

Accepted answers

ArborRose

If I grasp the question correctly…

If you are setting up an API, you want to go to the Domo AppStore and find the proper API connector. Type JSON in the search. Something like the JSON no code connector. In the connector you will input your settings like the URL you show. "updateMethod" looks like a parameter. You should find fields for entering parameters for header, body, or date.

Complete the form. Set your data structure and schedule. It will return a dataset. Partitioning is normally done using an ETL after you have the dataset from the API.

All comments

GrantStowell

Hi @mroker, could you further clarify the issue you are trying to solve for?

ArborRose

mroker

Thanks @ArborRose . So yes, I didn't know it was that easy in the ETL. Basically I wanted to use the API endpoint to partition the way Workbench does. I wanted to push data into Domo and if there was overlapping data, for it not to be duplicated. Also, if there was additional data, to append. So now what I am doing to solve that is to just use the replace on the API endpoint and I've created a ETL where the output is using a partition. It works perfectly.

ArborRose

@mroker - Glad to hear it's working.

After you get your API configured, I suggest making sure the data structure matches your data. The JSON connectors will try to guess at what your data is. If your field holds a date, make sure its a date. Not a string, not datetime…a date. Make sure joining identifiers come across as string or whatever you need. Otherwise, you'll kill yourself later trying to convert all those fields to cast them into something they should have been from the start.

With the No Code oAuth Connector, you can adjust the schema after its predicted the field types.