Datasets
Datasets
Duplicate Key Limit Error in Magic ETL
I am using the Magic ETL tool. I am trying to left join a DataSet with 4.5m records to a smaller table with ~1100 rows. After awhile, it crashes and I get the following error message:
"OnboardFlowExecution(10847) data flow execution id (ca085a327ae6417685896fb90f62e971) failed: MergeJoin: Duplicate key limit in left table exceeded! Limit: 10000"
How could I remedy this in order to successfully execute my join in the Magic ETL? Please advise. Thank you.
-Mark
Best Answer
-
Have you tried to swap the table positions in your join? Currently there is a limitation in Magic ETL Beta that the table on the left can not have more than 10000 duplicate values.
If the table on the right has less duplicate values try moving that table to the left side and changing the join direction as well.
Please let me know if this helps.
Thanks!
I work for Domo.
**Say "Thanks" by clicking the thumbs up in the post that helped you.
**Please mark the post that solves your problem as "Accepted Solution"10
Answers
-
I am having the same issue and same error message:
OnboardFlowExecution(74) data flow execution id (01323331e23d4970a0e69e69fbf861ee) failed: MergeJoin: Duplicate key limit in left table exceeded! Limit: 10000
Mark, if you found a solution could you please share?
Thanks
Jon
0 -
Have you tried to swap the table positions in your join? Currently there is a limitation in Magic ETL Beta that the table on the left can not have more than 10000 duplicate values.
If the table on the right has less duplicate values try moving that table to the left side and changing the join direction as well.
Please let me know if this helps.
Thanks!
I work for Domo.
**Say "Thanks" by clicking the thumbs up in the post that helped you.
**Please mark the post that solves your problem as "Accepted Solution"10 -
That worked. Thanks a million! I watched the videos but couldn't find that limitiaion.
Thanks for replying with a solution.
Jon
1 -
@markrenson, did Dark_Knight's also help solve your issue?
0 -
Swapping the table from left to right fixed the issue.
2
Categories
- All Categories
- 1.9K Product Ideas
- 1.9K Ideas Exchange
- 1.6K Connect
- 1.3K Connectors
- 306 Workbench
- 6 Cloud Amplifier
- 9 Federated
- 3K Transform
- 112 SQL DataFlows
- 649 Datasets
- 2.2K Magic ETL
- 4K Visualize
- 2.5K Charting
- 789 Beast Mode
- 78 App Studio
- 43 Variables
- 745 Automate
- 187 Apps
- 475 APIs & Domo Developer
- 67 Workflows
- 16 DomoAI
- 40 Predict
- 17 Jupyter Workspaces
- 23 R & Python Tiles
- 406 Distribute
- 117 Domo Everywhere
- 279 Scheduled Reports
- 10 Software Integrations
- 139 Manage
- 136 Governance & Security
- 8 Domo Community Gallery
- 46 Product Releases
- 12 Domo University
- 5.4K Community Forums
- 41 Getting Started
- 31 Community Member Introductions
- 113 Community Announcements
- 4.8K Archive