Duplicate Key Limit Error in Magic ETL

User_1826
User_1826 Member
edited March 2023 in Datasets

I am using the Magic ETL tool.  I am trying to left join a DataSet with 4.5m records to a smaller table with ~1100 rows.  After awhile, it crashes and I get the following error message:

 

"OnboardFlowExecution(10847) data flow execution id (ca085a327ae6417685896fb90f62e971) failed: MergeJoin: Duplicate key limit in left table exceeded! Limit: 10000"

 

How could I remedy this in order to successfully execute my join in the Magic ETL?  Please advise.  Thank you.

 

-Mark

Best Answer

Answers

  • I am having the same issue and same error message:

     

    OnboardFlowExecution(74) data flow execution id (01323331e23d4970a0e69e69fbf861ee) failed: MergeJoin: Duplicate key limit in left table exceeded! Limit: 10000

     

    Mark, if you found a solution could you please share?

     

    Thanks

     

    Jon

  • That worked.  Thanks a million!  I watched the videos but couldn't find that limitiaion.  

     

    Thanks for replying with a solution.

     

    Jon

  • kshah008
    kshah008 Contributor

    @markrenson, did Dark_Knight's also help solve your issue?

  • Swapping the table from left to right fixed the issue.