Import multiple files into single Dataset
Every day a new Excel file is dropped into a shared network folder. Every file is in the same format, and each file contains one day's data. Is there a way, using Workbench or any other method, to transport all Excel files from one local folder into a single Domo DataSet? Since a new file is added every day, every file will have a unique name.
Thanks!
Best Answers
-
Developer.domo.com is your central resource for all that special sauce.
https://developer.domo.com/docs/workbench-plugins/workbench-plugins in particular is where you want to start with plugins.
Aaron
MajorDomo @ Merit Medical
**Say "Thanks" by clicking the heart in the post that helped you.
**Please mark the post that solves your problem by clicking on "Accept as Solution"1 -
I just learned of a simple solution that works only if the source files are in CSV format.
If you create a new DataSet Job for Type CSV, you can specify the source as a folder rather than a file. Then as long as all of the files within the folder are CSVs, and all have the same fields, then they are all imported as if they were a single file, and there's no need to specify any file names.
5
Answers
-
Hi @nitot
I'm not aware of any default functionality that will accomplish this for you. However, you can definitely use Workbench and with a little coding you can create a plugin that can parse all those files and combine them into one.
When I worked for Domo we had a Fortune 100 client who had distributed data gathering across the globe with all the thousands of files stored in a single location in hundreds of subfolders. We built a Workbench plugin to traverse all of that structure and append like-tables together and upload them into a single dataset in Domo. It was awesome. Your request is definitely possible.
Aaron
MajorDomo @ Merit Medical
**Say "Thanks" by clicking the heart in the post that helped you.
**Please mark the post that solves your problem by clicking on "Accept as Solution"3 -
Thanks AS, are you aware of any resorces for getting started with these types of Plugins?
0 -
Developer.domo.com is your central resource for all that special sauce.
https://developer.domo.com/docs/workbench-plugins/workbench-plugins in particular is where you want to start with plugins.
Aaron
MajorDomo @ Merit Medical
**Say "Thanks" by clicking the heart in the post that helped you.
**Please mark the post that solves your problem by clicking on "Accept as Solution"1 -
I just learned of a simple solution that works only if the source files are in CSV format.
If you create a new DataSet Job for Type CSV, you can specify the source as a folder rather than a file. Then as long as all of the files within the folder are CSVs, and all have the same fields, then they are all imported as if they were a single file, and there's no need to specify any file names.
5 -
In my scenario, a box folder has separate csv files for every month and they all have the same columns and data types. I wanted to create one dataset in domo that will append all these files and give me an output and also pickup any new files that is added every month. But when I create a dataset using "Box advanced Connector" and specified to choose the folder and pick files with same name, the output is only coming out for 1 file and not from all the files. Can you please help me with this. You can see the image below as well.
Snip 1 shows the options in the connector and snip 2 shows the data only for October and does not show for previous months. The box folder that I want to connect is Headline/Ingress. I have given the same name in the options as well.
0
Categories
- All Categories
- 1.8K Product Ideas
- 1.8K Ideas Exchange
- 1.5K Connect
- 1.2K Connectors
- 300 Workbench
- 6 Cloud Amplifier
- 9 Federated
- 2.9K Transform
- 100 SQL DataFlows
- 618 Datasets
- 2.2K Magic ETL
- 3.9K Visualize
- 2.5K Charting
- 741 Beast Mode
- 58 App Studio
- 40 Variables
- 685 Automate
- 176 Apps
- 452 APIs & Domo Developer
- 47 Workflows
- 10 DomoAI
- 36 Predict
- 15 Jupyter Workspaces
- 21 R & Python Tiles
- 395 Distribute
- 113 Domo Everywhere
- 276 Scheduled Reports
- 6 Software Integrations
- 124 Manage
- 121 Governance & Security
- 8 Domo Community Gallery
- 38 Product Releases
- 10 Domo University
- 5.4K Community Forums
- 40 Getting Started
- 30 Community Member Introductions
- 108 Community Announcements
- 4.8K Archive