API using partition

mroker · 2024-02-14T12:01:37+00:00

There was an error rendering this rich post.

I have a question. I am using APIs now and I would like to know if it is possible to do partitioning using APIs. I know using Magic ETL and using Workbench you can do it, but can you do it using APIs? If I can't is the next step doing recursion?

The reason for asking is that I am uploading data daily and would like to add to it, and if any of the data is already there, would like to append to it, but not duplicate.

I don't see PyDomo having this feature so far.

Find more posts tagged with

Accepted answers

ArborRose

I don't believe you could do it in the API. As you state, you can implement that after receiving the dataset. I would think it would rely on the source of the API implementing the partition.

All comments

ArborRose

I don't believe you could do it in the API. As you state, you can implement that after receiving the dataset. I would think it would rely on the source of the API implementing the partition.

mroker

Thank you @ArborRose for the answer. I will then use the replace daily and use partitioning on the MagicETL.

ArborRose

@mroker - have a look in recent discussions for my question on "Effective Dating". This sounds exactly like what I am doing and the solution I created appears to work. My dataset is coming from an API scheduled daily. I append any records that are new or change to create an archive.