JSON No Code Connector Pagination with Sam.gov

sdickson
sdickson Member
in Connectors

The sam.gov API supports JSON pagination, but it's not clear if their API is supported with Domo's pagination options.

Sam.gov does include a next link, but it includes a placeholder for the api key, which needs to be replaced. Domo adds the configured api key as another parameter, but don't see how to configure Domo to remove the placeholder value, so Sam.gov rejects these requests.
"nextLink": "https://api.sam.gov/entity-information/v3/entities?api_key=REPLACE_WITH_API_KEY&page=1&size=10&primaryNaics=541512&includeSections=entityRegistration"


Alternatively, Domo's paging option allows setting of both page and size parameters, but Domo apparently expects or requires a total pages count in the result set Sam.gov returns totalRecords, not totalPages. I've tried to use totalRecords for total pages, but hit a maximum allow usage error from Sam.gov, which I suspect is the sam.gov 1000 requests/day.

Best Answer

  • ggenovese
    ggenovese Contributor
    Answer ✓

    You might want to try selecting the "Offset by pages" option and then "No" in the pagination configuration, where it asks if the data contains the total number of pages, that has worked for me in the past. This works if you get a NULL response when you get to the last page, the connector will stop trying to keep fetching pages. If the api throws an error however, then this may not be a good fit for the JSON No Code connector, and you may need to look into using Domo Jupyter Workspaces instead. Regarding the Rate Limit, there is no "per day" rate limit that the connector allows just seconds, minutes, and hours.

    image.png

Answers

  • ggenovese
    ggenovese Contributor
    Answer ✓

    You might want to try selecting the "Offset by pages" option and then "No" in the pagination configuration, where it asks if the data contains the total number of pages, that has worked for me in the past. This works if you get a NULL response when you get to the last page, the connector will stop trying to keep fetching pages. If the api throws an error however, then this may not be a good fit for the JSON No Code connector, and you may need to look into using Domo Jupyter Workspaces instead. Regarding the Rate Limit, there is no "per day" rate limit that the connector allows just seconds, minutes, and hours.

    image.png

Categories