Pydomo Stream API upload_part skipping files?
I am using pydomo to upload several gzipped files in parallel. (These gzipped csv files are parts of one large SQL table, broken approximately evenly.) However, once the execution is commited, there seems to be fewer rows than the original table. The difference in the number of rows seems to amount to the number of rows in one of the files meant to be uploaded.
Just like in the stream api example, I create an stream, I create an execution for that stream, and I use streams.upload_part() with the created stream and execution. Is there a way to return the status of streams.upload_part(), so that I know which files have failed?
Thanks!
Comments
-
Not sure abut the issue specifically, but our company open sourced a python script we wrote to use the streaming API. Maybe there's some useful code in there, or feel free to use it as is.
0
Categories
- All Categories
- 1.2K Product Ideas
- 1.2K Ideas Exchange
- 1.4K Connect
- 1.1K Connectors
- 273 Workbench
- 2 Cloud Amplifier
- 3 Federated
- 2.7K Transform
- 78 SQL DataFlows
- 526 Datasets
- 2.1K Magic ETL
- 2.9K Visualize
- 2.2K Charting
- 440 Beast Mode
- 23 Variables
- 514 Automate
- 115 Apps
- 391 APIs & Domo Developer
- 8 Workflows
- 26 Predict
- 10 Jupyter Workspaces
- 16 R & Python Tiles
- 332 Distribute
- 77 Domo Everywhere
- 255 Scheduled Reports
- 67 Manage
- 67 Governance & Security
- 1 Product Release Questions
- Community Forums
- 40 Getting Started
- 26 Community Member Introductions
- 68 Community Announcements
- 4.8K Archive