JSON Dynamic Body

skeets20

I'm liking the JSON Low Code OAuth connector, especially the dynamic parameters and looping option. Problem is, we need to post a body with dynamic values. Seem like the dynamic parameter capability could be extended to do token replacement in the body content as well.

ArborRose

I liked the JSON OAuth connector also, but dislike the speed. I found that using a different method with Jupyter notebooks and Python, I have more flexibility talking to my APIs and my speed is significantly faster.

skeets20

Thanks, I looked at that as well but didn't see the "requests" library as an option. That's usually my goto for calling api's from Python. Which library in the Domo Python script task are you using?

ArborRose

The requests library is a popular choice for making API calls in Python, and it’s what I would typically use in most environments. However, when working within Domo's Python script task, the standard library might be more limited.

If requests isn't available directly in Domo, you can achieve similar functionality using the http.client module or even urllib from Python's standard library, which can handle HTTP requests fairly well, although with a bit more boilerplate code compared to requests.

import http.client

import json


conn = http.client.HTTPSConnection("api.example.com")


headers = {

    'Content-Type': "application/json",

    'Authorization': "Bearer your_access_token"

}


body = json.dumps({

    "dynamic_param_1": "value1",

    "dynamic_param_2": "value2"

})


conn.request("POST", "/your/api/endpoint", body, headers)


res = conn.getresponse()

data = res.read()


print(data.decode("utf-8"))

If you decide to go the Jupyter notebook route, you might also look into pandas for managing data and requests for more straightforward API calls. This combination offers speed and flexibility, making it a powerful alternative to using connectors directly in Domo.

skeets20

Thanks so much. Just went back to look at doing this. Noticed requests lib is in the Domo 2 environment which I must not have looked at last time. Now i'm looking into the docs to see if it's possible to leverage the existing "account" to handle authentication and fetching the token. Thanks for your help

skeets20

Turns out this may not be possible, using your method and the same python code I have working locally, both get an error. Based on what I'm reading in some other threads, Domo doesn't allow calls to URLs outside their environment. Unless I'm missing something, this is a bust as it seems Domo won't resolve external URLs in this scenario.

socket.gaierror: [Errno -3] Temporary failure in name resolution

My use case BTW, is that I have a third party data provider. I have to send a request to generate a dataset. They return an ID that I have to monitor status of and once completed, I get a payload of file ids for csv files that I want to import into domo and link with another dataset. Unfortunately, they require a POST body and I cant do a dynamic body in Domo JSON call.

ArborRose

Given your use case, where you need to handle a POST request with a dynamic body and interact with third-party services, you might want to explore a couple of alternatives:

External Server Proxy: Set up an external server or service that can act as a middleman. Your external server can handle the API requests, including authentication and dynamic POST bodies, and then push the data to Domo. This way, Domo only needs to interact with your external server, which can be configured to comply with Domo’s environment restrictions.

API Gateway: Consider using an API Gateway service (like AWS API Gateway or similar). You can configure it to handle authentication and data processing, then route the necessary data to Domo. This approach can also help manage API rate limits and handle complex authentication flows.

Local Pre-Processing: As an interim solution, you could continue processing the API calls and handling dynamic requests locally (using your Jupyter notebooks and requests library) and then upload the processed data to Domo. This way, you leverage Domo for visualization and analysis while using local tools for API interactions.

skeets20

I did end up going with the API Gateway route with a lambda function. In addition to not allowing tokenized bodies, Domo can't download a csv file to dataset using the oauth connector. My lambda goes through all the API calls, downloads the csv to pandas and then returns that through the gateway as json. Works great but not ideal since we are trying to deprecate lambdas in favor of Domo. Thanks for all you suggestions @ArborRose