Workbench 5 TXT File upload
Trying to upload a TXT file to Domo that is 301,321 KB and 1.7M rows, anyone have any tips? I can't seem to make any connectors work. They all give me the same error that a file cannot be larger than 254MB. Also, has anyone tried to use the Access app? IT asks for SFTP Hostname for a Microsoft Access connector and the port number?
Best Answer
-
A couple of things to try:
- TXT files compress really well (assuming you mean a CSV file) . If you put it in a zip file, you can use an email connector or a dropbox connector to send in the zip file, which Domo then auto-extracts the CSV file from
- There's also the 'legacy' file upload connector, that uses a different method/library to do the upload (the one that shows an upload progress bar) and potentially doesn't have that issue. You'll find it here (insert your domo domain name): https://{your_domo_domain}.domo.com/appstore/connectors/com.domo.connector.fileupload.legacy?origin=dc&classification=UPLOAD
- Finally, our company developed a python script to stream files up into Domo using the 'straming API' but it's a lot more complicated to use. You'll find the open-source code here: https://github.com/BlueBikeSolutions/domo-stream-uploader
Good luck (I reckon the zipped file is the easiest approach)
P.S - Workbench definitely shouldn't have those constraints...not sure why you'd hit limits with that approach
1
Answers
-
A couple of things to try:
- TXT files compress really well (assuming you mean a CSV file) . If you put it in a zip file, you can use an email connector or a dropbox connector to send in the zip file, which Domo then auto-extracts the CSV file from
- There's also the 'legacy' file upload connector, that uses a different method/library to do the upload (the one that shows an upload progress bar) and potentially doesn't have that issue. You'll find it here (insert your domo domain name): https://{your_domo_domain}.domo.com/appstore/connectors/com.domo.connector.fileupload.legacy?origin=dc&classification=UPLOAD
- Finally, our company developed a python script to stream files up into Domo using the 'straming API' but it's a lot more complicated to use. You'll find the open-source code here: https://github.com/BlueBikeSolutions/domo-stream-uploader
Good luck (I reckon the zipped file is the easiest approach)
P.S - Workbench definitely shouldn't have those constraints...not sure why you'd hit limits with that approach
1 -
I am exporting a large table from Access TXT, then I saved as csv file. Once I put it on my desktop rather than a shared drive, I was able to use workbench and do a local file. I am going to try to do the Zip File option as well, because I think thats a really clever way to do this sort of task in the future! Thank you so much!
0
Categories
- All Categories
- Product Ideas
- 2.1K Ideas Exchange
- Data Connections
- 1.3K Connectors
- 309 Workbench
- 17 Cloud Integrations
- Data & ETL
- 2.3K Magic ETL
- 120 SQL DataFlows
- 667 Datasets
- Visualize & Apps
- 90 App Studio
- 198 Pro-code Components
- 2.6K Charting & Analyzer
- 873 Calculations & Variables (Beast Mode)
- AI & Data science
- 23 Domo AI & AI Chat
- 4 Managing AI
- 18 Jupyter Workspaces
- Automate
- 122 Workflows
- Alerts
- Distribute
- 118 Domo Everywhere
- 284 Reporting
- Manage
- 146 Governance & Security
- 489 APIs
- 11 Add-ins & Plugins
- 13 Domo Community Gallery
- 49 Product Releases
- 13 Domo University
- Community Forums
- 41 Getting Started
- 31 Community Member Introductions
- 118 Community Announcements
- 5K Archive