Workbench 5 TXT File upload
Trying to upload a TXT file to Domo that is 301,321 KB and 1.7M rows, anyone have any tips? I can't seem to make any connectors work. They all give me the same error that a file cannot be larger than 254MB. Also, has anyone tried to use the Access app? IT asks for SFTP Hostname for a Microsoft Access connector and the port number?
Best Answer
-
A couple of things to try:
- TXT files compress really well (assuming you mean a CSV file) . If you put it in a zip file, you can use an email connector or a dropbox connector to send in the zip file, which Domo then auto-extracts the CSV file from
- There's also the 'legacy' file upload connector, that uses a different method/library to do the upload (the one that shows an upload progress bar) and potentially doesn't have that issue. You'll find it here (insert your domo domain name): https://{your_domo_domain}.domo.com/appstore/connectors/com.domo.connector.fileupload.legacy?origin=dc&classification=UPLOAD
- Finally, our company developed a python script to stream files up into Domo using the 'straming API' but it's a lot more complicated to use. You'll find the open-source code here: https://github.com/BlueBikeSolutions/domo-stream-uploader
Good luck (I reckon the zipped file is the easiest approach)
P.S - Workbench definitely shouldn't have those constraints...not sure why you'd hit limits with that approach
1
Answers
-
A couple of things to try:
- TXT files compress really well (assuming you mean a CSV file) . If you put it in a zip file, you can use an email connector or a dropbox connector to send in the zip file, which Domo then auto-extracts the CSV file from
- There's also the 'legacy' file upload connector, that uses a different method/library to do the upload (the one that shows an upload progress bar) and potentially doesn't have that issue. You'll find it here (insert your domo domain name): https://{your_domo_domain}.domo.com/appstore/connectors/com.domo.connector.fileupload.legacy?origin=dc&classification=UPLOAD
- Finally, our company developed a python script to stream files up into Domo using the 'straming API' but it's a lot more complicated to use. You'll find the open-source code here: https://github.com/BlueBikeSolutions/domo-stream-uploader
Good luck (I reckon the zipped file is the easiest approach)
P.S - Workbench definitely shouldn't have those constraints...not sure why you'd hit limits with that approach
1 -
I am exporting a large table from Access TXT, then I saved as csv file. Once I put it on my desktop rather than a shared drive, I was able to use workbench and do a local file. I am going to try to do the Zip File option as well, because I think thats a really clever way to do this sort of task in the future! Thank you so much!
0
Categories
- 10.5K All Categories
- 3 Connect
- 913 Connectors
- 250 Workbench
- 458 Transform
- 1.7K Magic ETL
- 69 SQL DataFlows
- 476 Datasets
- 183 Visualize
- 249 Beast Mode
- 2.1K Charting
- 11 Variables
- 16 Automate
- 354 APIs & Domo Developer
- 88 Apps
- 3 Workflows
- 20 Predict
- 5 Jupyter Workspaces
- 15 R & Python Tiles
- 245 Distribute
- 62 Domo Everywhere
- 242 Scheduled Reports
- 20 Manage
- 41 Governance & Security
- 168 Product Ideas
- 1.2K Ideas Exchange
- 9 Community Forums
- 27 Getting Started
- 14 Community Member Introductions
- 55 Community News
- 4.5K Archive