Microsoft SharePoint Online Rest Connector - Download all files in a folder

mhouston

I've gotten the Microsoft SharePoint Online Rest Connector connected in our Domo instance and have been succesful pulling in individual file contents using the "Download Item Contents" report option.

 

However, we have a sharepoint folder where users will drop multiple files, and I need to bring the contents of all of these files into Domo. I'd like to avoid having to set up a connector for each individual file, especially as I do not always know when a new file has been added.

 

Is there  a way to use the Sharepoint connector to download all files in a sharepoint folder, rather than having to point to an individual file?

GrantSmith

Hi @mhouston 

Unfortunately it appears that their API only supports pulling an individual file and not an entire directory. If you're wanting to programmatically pull all files in the directory you options would likely be either write a script to query all of the files in the folder then call the API individually for each file or possibly write a custom connector to do this for you.

 

However, the concern is that if an entire directory was able to be uploaded there's no real enforcement that the data structures are the same and then you get into the issue of how to handle the differences when combining them into the same dataset.

 

Your options are either 

- Keep it as is and manually add a new connection for each new file that gets added ensuring the data structures are consistent (single file to a new dataset makes it consistent)

- Write a custom connector or script to read the entire data folder and somehow handle the possible data structure differences when uploading to the same dataset which is a much more complex option.

jaeW_at_Onyx

could also write a script that generates a connector based on al ist of values in a CSV.

 

but at that point why not sync Sharepoint with a local folder and then use workbench?

mhouston

Thank you both for your insights. Unfortunately I need some of the sharpeoint metadata associated with the files so just syncing with a local folder and using workbench doesn't provide the information I need. I know the data are correctly formatted because we have a template based process, so Ill explore the use of a script to provide the solution I need.