Import multiple files into single Dataset

nitot

Every day a new Excel file is dropped into a shared network folder.  Every file is in the same format, and each file contains one day's data.  Is there a way, using Workbench or any other method, to transport all Excel files from one local folder into a single Domo DataSet?  Since a new file is added every day, every file will have a unique name.

Thanks!

AS

Developer.domo.com is your central resource for all that special sauce.

https://developer.domo.com/docs/workbench-plugins/workbench-plugins in particular is where you want to start with plugins.

nitot

I just learned of a simple solution that works only if the source files are in CSV format.

If you create a new DataSet Job for Type CSV, you can specify the source as a folder rather than a file.  Then as long as all of the files within the folder are CSVs, and all have the same fields, then they are all imported as if they were a single file, and there's no need to specify any file names.

 

AS

Hi @nitot

 

I'm not aware of any default functionality that will accomplish this for you.  However, you can definitely use Workbench and with a little coding you can create a plugin that can parse all those files and combine them into one.

When I worked for Domo we had a Fortune 100 client who had distributed data gathering across the globe with all the thousands of files stored in a single location in hundreds of subfolders.  We built a Workbench plugin to traverse all of that structure and append like-tables together and upload them into a single dataset in Domo.  It was awesome.  Your request is definitely possible.

nitot

Thanks AS, are you aware of any resorces for getting started with these types of Plugins?

AS

Developer.domo.com is your central resource for all that special sauce.

https://developer.domo.com/docs/workbench-plugins/workbench-plugins in particular is where you want to start with plugins.

nitot

I just learned of a simple solution that works only if the source files are in CSV format.

If you create a new DataSet Job for Type CSV, you can specify the source as a folder rather than a file.  Then as long as all of the files within the folder are CSVs, and all have the same fields, then they are all imported as if they were a single file, and there's no need to specify any file names.

 

user053304

In my scenario, a box folder has separate csv files for every month and they all have the same columns and data types. I wanted to create one dataset in domo that will append all these files and give me an output and also pickup any new files that is added every month. But when I create a dataset using "Box advanced Connector" and specified to choose the folder and pick files with same name, the output is only coming out for 1 file and not from all the files. Can you please help me with this. You can see the image below as well. 

 

Snip 1 shows the options in the connector and snip 2 shows the data only for October and does not show for previous months. The box folder that I want to connect is Headline/Ingress. I have given the same name in the options as well.