Using Data Joins - Limits to Join Data?

Options
jlafee
jlafee Member
in Archive

Hi Everyone,

 

   I have a data set of about 400K rows of sales data.  It includes our item code, but no other details.  I'm trying to match it to our item master, which has about 2K rows to fill in the blanks.  I know there is a limitation to the left dataset in a join, I believe it's 10K rows, so I'm using the item list on the left and the sales data on the right.  In both tables, the item code is text, and Domo agrees on the data type.

 

However, when I run the dataflow (in preview or in full) it does not join the data.  I get 400K rows of sales data with blank item columns where the extra details should be.  Are there any limits or best practices to guaranteeing two text columns match up?

 

Thank you!

Comments

  • [Deleted User]
    [Deleted User] Member

    Thx @jlafee!

     

    Check out this article, we are looking in to the limitation question you posted. One of my top guys will circle back shortly.

     

    Regards,

     

     

  • jlafee
    jlafee Member

    Thanks, DaniBoy.  I've been through that document quite a few times, and so I don't believe there is anything wrong with the way the join is set up.  I think there is some just rule, quirk, or missing step in preparing the data that is preventing the final join.  

  • nicolasfeddern
    nicolasfeddern Domo Employee

     

    Are you using Magic to do your left join?

     

     

  • jlafee
    jlafee Member

    Yes, @nicolasfeddern, this is all built in Magic.  I've done it as a left outer and right outer to see if that helps, both give the same results except for moving the location of the blank columns on the dataset.

  • nicolasfeddern
    nicolasfeddern Domo Employee

    Have you tried swapping the inputs and changing from a left to a right join? See the attached photo - your biggest dataset should be the right side of the join. 

     

    Let me know if that works for you.

  • jlafee
    jlafee Member

    @nicolasfeddern, you and I think a lot alike.  I have the 400K dataset on the right, and the 2K on the left on a right join.  Still having the same issue, but I appreciate the confirmation that I at least have it set up correctly.

  • nicolasfeddern
    nicolasfeddern Domo Employee

    I see - have you confirmed that the datatype of the columns in the join condition are the same? That might be the cause of the blanks. Could you share a sample of the data that you're attmepting to join or take a screenshot of your Magic ETL join?

  • nicolasfeddern
    nicolasfeddern Domo Employee

    Looks like you already checked on the column data types. Do you have any trailing or leading spaces that might cause a failure to match between the join keys?  

  • Unknown
    Unknown

    @jlafee, tagging you to check out @nicolasfeddern's reply.

Categories