Missing Pagination Query Params for JSON Connector

ericziengsGB

Hi

I am trying to add pagination to a dataset using the JSON Connector. Domo will show a nice preview URL with the pagination query parameters added to the URL, but when I press 'test' and view our server logs, the API call is made without pagination parameters. If I hardcode query parameters under Settings → Data Selection → Advanced, I can see those in our server logs, but the supposed automatic pagination query parameters are never included.

Has anyone experienced something similar?

It's of course possible that our API service is dropping the parameters, but I think it's highly unlikely - if I make the same request from cURL or Postman, I can successfully use the pagination query parameters.

I have tried both with and without the 'Encode query parameter keys/values' settings, but nothing works.

Thank you for taking the time to read this, any help is much appreciated!

GrantSmith

Is this only when you do the test of the connector or have you actually run the connector and it's requesting the same data multiple times?

ericziengsGB

Hi Grant

When I run the connector, I realized the pagination actually works. It is only under Settings → Parsing, both 'Pagination' and 'Parsing' tabs, when Domo makes an API call to fetch some data to preview in the browser, that the pagination query parameters are not included. In this case, it is a bit confusing - especially since there is a 'test' button next to a URL with pagination query parameters added, but when that 'test' button is pressed, a request is sent without pagination query parameters.

In our specific case, for backwards compatibility reasons, we have to return every single entry if no pagination query parameters are supplied, and this results in a "Response size too large" error for the endpoint in question, making the 'Parsing' settings impossible to use as data cannot be loaded.

Fortunately we were able to work around it by hard coding pagination query parameters under Data Selection → Advanced, then doing the configurations under 'Parsing', and then removing the hard coded headers.

GrantSmith

That's what I was curious about. It can be a bit confusing but glad you got it working properly.