Tutorial: build your own connector against petfinder API

jaeW_at_Onyx

Building Custom Connectors in Domo.

• Read the KB article in developer.domo.com then click Build Now and authenticate

Upload Icons

• I found images using google searches, then resized them using resizeimage.net to the appropriate dimensions.

 

User Authentication

• take note of the Configure Authentication KB article

Although OAuth2.0 is a virtually universal standard, every API will have slightly different implementations and parameters, so familiarize yourself with the vendor's developer documentation. Authentication is arguably the hardest part of building a connector.

 

Pro Tip - If you're running into roadblocks, try other tools to make sure you're structuring the request properly

Tools you can use for troubleshooting include:

• Postman
• Running a CURL command in Visual Studio Code
• web-based tools like curl_to_fetch to convert a curl request to a vanilla JavaScript request.

RTFM!

Regardless your troubleshooting tool of choice, at the end of the day, you must generate javascript that the IDE can recognize. Review the overview section of process-data:

1. all your javascript must be ES5 compatible (it's an older version of JS) AND
2. you explicitly CANNOT use XMLHttpRequest. Instead you're guided to use httprequest.
3. console.log has been replaced with DOMO.log

Back to the Connector

For petfinder.com, you'll need a petfinder developer account before you'll receive a client_id and secret. The developer documentation gives a sample curl request for receiving an access token.

Under User Authentication, you configure how your connector authenticates against the source API. Regardless of which authentication method selected, even if there is no authentication required, you must 'tell' the connector that you successfully completed authentication.

auth.authenticationSuccess();

Define the appropriate fields users must populate to authenticate. You'll see these fields again when you configure a new connection in the Domo Data Center.

The this script will request an access token from petfinder.com and if a valid access token is successfully retrieved, tell Domo that the operation was a success.

// get variables from the form
client_id = metadata.account.client_id
client_secret = metadata.account.client_secret

// url and body dictated by vendor specification.
token_url = '<https://api.petfinder.com/v2/oauth2/token>'
token_body = `grant_type=client_credentials&client_id=${client_id}&client_secret=${client_secret}`

// store the results of your API reuest, a string, in a variable 'res'
var res = httprequest.post(token_url, token_body);  

//parse the string, res, into a JSON object, data 
var data = JSON.parse(res)

//navigate data and store access_token into a new variable 
var token = data.access_token

//if token exists, call the 'success' method else throw an error
if(token){
  auth.authenticationSuccess();
}
else{
  auth.authenticationFailed('Your client_id and secret are incorrect');
}

Define the Report

Because we used a custom authentication method, Domo did not store the access token. So we'll request another one. Then we request a report, in this case page 2 of the dogs report.

This code could be modified to allow users to select from a drop-down of animal types. Or limit the number of results returned.

client_id = metadata.account.client_id
client_secret = metadata.account.client_secret

token_url = '<https://api.petfinder.com/v2/oauth2/token>'
token_body = `grant_type=client_credentials&client_id=${client_id}&client_secret=${client_secret}`;

var res = httprequest.post(token_url, token_body);
var data = JSON.parse(res)
var token = data.access_token

// create a new httprequest for the dog report
// authenticate against the API by including token in the request header
var report_url = '<https://api.petfinder.com/v2/animals?type=dog&page=2>'
httprequest.clearHeaders(report_url)
httprequest.addHeader('Authorization', 'Bearer ' + token);

res = httprequest.get( report_url)
data = JSON.parse(res)

// send the data.animals array to Domo as a dataset
datagrid.magicParseJSON(data.animals)

Warning

Be wary of magicParse as it will generate a schema based on data extracted from a given run from the API. Because the schema is dynamically generated, between runs the output table in Domo could change which could impact ETLs that expect a given format or set of columns.

For downstream ETL stability, a better approach may be to parse the data extracted from the API and explicitly define a fixed schema; however that's beyond the scope of this tutorial.

Heidi

This is very helpful. Thanks for putting this together!